Uncategorized

Efergy E-Max Wireless Energy Monitor from Reduction Revolution

December 19, 2024 by Simon

This blog post will review this Energy monitor from Reduction Revolution.

Background

Last year we bought a Power Pal Energy Monitor Bluetooth Power Pal Energy Monitor. This battery powered device attaches to our house power box smart meter to measure power usage in realtime.

It works by detecting the LED flash on the Energy Provider Smart Meter (power used) and reports the power usage to a Power Pal mobile app. If my phone is away from the house it catches up on the power usage when returning to the house.

The Power pal app on my phone saves this power usage stats and graphs the usage over time. The Power Pal device has been syncing to my phone (via Bluetooth) over the last 12 months, this has been helpful to allow is it manually monitoring and track our power usage.

We were able to track our house power 24/7 and measure what was using the most power in realtime. We could tell quite quickly that our air conditioner was consuming 70% of our power.

After we got an EV we wanted to track power usage into the house. After a few months we switched to Origin Energy’s EV Power Up program where we plug in the car when we are not using itand Origin decide when to charge to meet our requested charger limit (at cheaper power rates).

With EV Power Up Origin will try and use surplus renewable energy during the day or off peak energy at night during off peak times (all at 8c per kW/h). In return we do not expect to charge during the peak demand when cheap power is not available.

The device could measure and chart power usage over 24 hours.

On a typical month the car (EV) uses 8kW of power a day. We plug in ever day to remain topped up to 80% battery capacity ensure we have over 410km range everyday. The car is plugged into a 15A 240V Socket but we only pull 12A to be nice to the cabling. The car charger happily lowers the power drawn if the house is using to much power or if the charger, charger cable or charger plug is too hot.

The Power Pal app energy usage matches the cars power usage.

The Power Pal screenshot below shows monthly consumed data at the bottom of the following screenshot.

Our power usage is generally seasonal (summer and winter high load), we have been able to track down high power usage patterns outside the demands of heating and cooling. We learned that our dish washer uses a lot of power in certain modes.

We have reduced our power usage by about 40%, but we want to save mor. This is where the Efergy E-Max Power Monitor with colour display comes in, as it has an always on display to remind the whole house how much power is being used.

Efergy E-Max Wireless Energy Monitor with Colour Screen

Package Contents

Manual

Colour display and battery compartment

Battery compartment at the back of the screen

Transmitter box and batteries

This is the single phase power clamp that clips over the power line in the meter box.

The colour display is bright

The colour display only draws out 1W of power.

Our electrician installed the clamp on the main incoming power line (after the fuse and energy meter and before the switch board).

The energy monitor can be installed on any circuit you want (e.g on a Pool Pump, solar circuit, Car charger or all power).

I connected the energy monitor screen to the transmitter before talking it inside.

I setup the Energy Monitor near our Television in the lounge room.

The screen updates every 10 seconds with realtime power usage.

Configuring the Energy Monitor

The manual clearly explains the process of configuring the device.

I held down the Mode Set button for 3 seconds until I was asked to set the Date Time. I entered the date and time. I then proceeded to set the..

Voltage: 240V
Currency: $
Tariff 1 – Rate: 0.34c kW/h (I checked my bill)
Tariff 1 – Time Start and End: 7:00 ~ 22:00
Tariff 2 – Rate: 0.14c kW/h
Tariff 2 – Start and End: 22:00 ~ 07:00
Tariff 3 – Rate: N/A
Tariff 3 – Start and End: N/A
Tariff 4 – Rate: N/A
Tariff 4 – Start and End: N/A
Budget – Per Month: $400
Alarm: 12kW
Footprint: 0.5 kgCO2
Calibration: 0.98 (I determined this by following the manual and comparing the power usage to the Smart Meter over 24 hours)

After a few weeks

I have tested this over a few weeks.

At any time I can press the Next/Right button to cycle through the day/week or months power usage to see a report.

Pros

These are the upsides

No more logging into a Power Pal Mobile app to check house power usage.
The whole house can see our energy usage, we have made a game of lowering power usage.
This a device has helped educated my kids on our energy footprint/usage.
We no longer blindly adjust the AC up or down (without looking at power usage).
Zero interaction once connected.

Cons

These are the downsides

You will need an electrician to install the Energy Minute (5 minute job)
The Tariff setup has no weekend awareness
The Tariff time windows do not align with the Smart Meter/Energy Providers bending of the power prices times, as they change prices depending on grid demand.
No ability to detect and subtract Origin EV Power Up power usage – The EV can pull power at 8c per kWh during the day or night. If wanted to track our EV’s power usage I would need to put a clamp on that circuit or add a second display.
You may want to calibrate the power usage to get accurate readings (see the manual)

Conclusion

We have definitely lowered our power usage by 40% just by seeing the current power usage and deciding to not turn something on until off peak times (dish washer, washing machine, AC etc).

The colour screen has batteries so the device can be unplugged and carried around the house to real power while I turn devices on and off. Now I know what uses the most power.

I hope Efergy create a screen with multiple clamps for monitoring multiple circuits.

Thanks for reading.

v1.0

Trimming 16GB memory usage in Windows 11

July 30, 2023 by Simon

I have been using the same Windows 11 Installation since 7/5/2022, and I have noticed that the memory usage was jumping up to 22GB of memory used when on the desktop. I had 32GB memory but upgraded to 64 GB.

Maybe I need to free some bloatware and free some memory?

Before

Before I attempt to free up bloatware on my system, here is a before state of memory and processes on my system. I used the Windows built-in task Manager to view resources.

23GB Memory Used

444 Processes, 9.3K Threads and 3.4 Million Handled

Removing Bloatware from Windows 11

Warning: No warranty is given to the steps below; you can easily break your computer if you remove or disable the wrong things. Do take a backup of your system and files.

I downloaded the Ultimate Windows Utility app from The Ultimate Windows Utility (christitus.com)

Image of the webpage for Ultimate Windows Tool

Tip found via

Tip for The Ultimate WIndowss Utility found via NortoDan

To run this, I ran the following PowerShell Command

iwr -useb https://christitus.com/win | iex

This is what the application looks like.

I clicked the Tweaks tab and then the Desktop button.

I ticked these settings that I knew were not going to break my system,.

Ultimage WIndows Tool Tweaks then Desktop Form

I clicked Run Tweaks button and rebooted my PC.

Nice, that’s over 100 processes eliminated after a reboot.

Windows task manager showing 12GB memory used, 313 active processes

Disabling Unwanted Windows Services

I opened the Windows Services as an Administrator.

From here I sorted by running services and disabled the

Use Right Click, then select Disable to see if memory usage goes down, then Double click the item, select Stop, and select StartUp Type Disabled to permanently diable the service on startup.

I did not disable critical services of system services.

Microsoft SysInternal Tools

I downloaded the awesome and free Microsoft SysInternals tools from Microsoft.
Sysinternals Suite – Sysinternals | Microsoft Learn

Download this and save it to e.g C:\Utils\SysInternals

The most common tools I use from this suite are

Process Monitor
Monitor file system, Registry, process, thread and DLL activity in real-time.

Disk Usage (DU)
View disk usage by directory.

TCPView
Active socket command-line viewer.

Process Explorer
Find out what files, registry keys and other objects processes have open, which DLLs they have loaded, and more. This uniquely powerful utility will even show you who owns each process.

PortMon
Monitor serial and parallel port activity with this advanced monitoring tool. It knows about all standard serial and parallel IOCTLs and even shows you a portion of the data being sent and received. Version 3.x has powerful new UI enhancements and advanced filtering capabilities.

Autoruns
See what programs are configured to startup automatically when your system boots and you login. Autoruns also shows you the full list of Registry and file locations where applications can configure auto-start settings.

Process Monitor
Monitor file system, Registry, process, thread and DLL activity in real-time.

SysInternal AutoRuns will show all the places that software can be started (at startup or user interaction with Windows). I disabled the software that I did not need. I was careful not to disable drivers or system applications.

I used SysInternal TPCView to see what software was using my Network, and I found a few rather clatty services associated with my backup software.

I disabled the Protection in the Acronis App but this had no affect on the background network adctivity

The older non-subscription-based Acronis True Image 2021 software I use has Ransomware protection that can not be disabled. The only way to disable it was to stop this “Acronis Active Protection (TM) Service).

I rebooted my machine, and active processes and threads were coming down. Memory usage was a lot lower.

Windows task manager showing 8.4GB memory used, 277 active processes

I re-ran SysInternal TCPView to see if the remaining apps were not chatty. Only my STreamdeck apps were talking to the network.

SysInternal Process Monitor is also awesome for seeing a real-time list of all activity of a system at that given time. This list can give insight into background activity that I did not realise was happening.

I disabled apps and services that I did not want running.

I then used SysInternals Process Explorer to show a tree view of apps running on my system.

Sorting the treeview by memory is useful.

I repeated all of the above until I had nothing left to disable or remove

Windows task manager showing 6.8GB memory used, 213 active processes

After

Before

23GB Memory Used

444 Processes, 9.3K Threads and 3.4 Million Handles

After

6.8GB Memory Used

213 Processes, 3.8K Threads and 1.3 Million Handles

That is not bad for an hours work.

Good luck

Using a 12-year-old dual Xeon server setup as a desktop PC (updated Video Card)

July 12, 2023 by Simon

Even since the mid 90’s I have alway wanted a dual processor/socket system. My first PC was an AMD K6-2 333 MHz (Single Core) processor and I have always loved building computers from parts.

Recently, I stumbled across Tech Yes City’s video about buying a 12 year old Xeon (server) processors an dusing it in a gaming machine..

I do not need a gaming PC but I do need a spare development PC and PC my kids can use for Homework.

I searched eBay, GumTree and AliExpress in vein for a cheap a X58 mainboard. I ended up finding a new (new really) Tyan S7012 (S7012GM4NR) mainbaiord for $119 USD.

Mainboard

NEW - Tyan S7012GM4NR Dual Intel Socket LGA1366 DDR3 Motherboard

I was mesmerized by the 18 Memory Slots and dual CPU sockets.

Memory

I orderd 12 x Hynix RDIMM PC3-10600R 1333Mhz (HMT351R7BFR8C) memory modules (48GB in total).

12 RDIMM Hynix 24GB (6x 4Gb) 2Rx8 PC3-10600R 1333Mhz (HMT351R7BFR8C)

I ordered 12 because I could not find a cheap set of 18 x RDIMM’s. I will try and find 6 more memory modules later.

I ordered 2x Intel Xeon E5620 SPEC SLBV4 for 99c. These chips are 4 core, 8 threads per processor.

Processor(s)

After some research, I realised these chips are way too slow.

At the same time, I accidentally won an auction for 2x 6 core/12 Thread Intel Xeon E5645 Processor (SLBWZ 2.4GHz 12M).

Matched Pair Intel Xeon E5645 Processor 6 Core CPU LGA 1366 SLBWZ 2.4GHz 12M

The more I researched, I realised I should purchase 2 of the fastest X5600 series processor possible. I purchased 2x X5690 Xeon processors from Ali Express.

Each processor is a 6 core/12 thread 3.46Ghz (single-core boost to 3.73Ghz), 12MB cache processor made with 32nm lithography.

1 hour after purchasing from Ali Express I has confirmation from the seller.

The board arrived in a few weeks, The board was in mint condition.

Heatsink(s)

I ordered2 S7012 compatible heatsinks (Noctua NH-U9DX i4)

Preparing the Processor(s)

I was a bit scared of the potential 2x 130W each processor could generate at 100% usage so I decided to lap the surface of each processor using 400 to 3000 grit sandpaper. I used a sheet of glass to lap each processor with a fresh wet/dry sandpaper sheet. I used Isopropot alcohol to lubricate the paper and carry away metal.

It turns out that each processor was very distorted. The IHS was bent out of shape.

lapping of theheqatsink shows the cou IHS was distorted.

I lapped each processor to remove each concave and convex shape.

Each processor IHS was very flat.

Case

I was lucky enough to get a Phanteks Enthoo Pro 2 D-RGB Tempered Glass Full-Tower ATX Case – Black case for $141

The Tyan S7012 Mainbioard is 30cm x 33cm and it will nt fit in a ATX case.

The installed Tyan S7012 case looks tiny when installed in the Phanteks case.

Power Supply

I ordered a Corsair RMx Series™ RM1000x — 1000 Watt 80 PLUS Gold Fully Modular ATX PSU as it has the 3x EPS12V power connectors that the Tyan S7120 requires.

Etc

I added a Sound card, USB 3 card, 12 fans.

I added a Wifi6 card

Installing Windows

I grabbed a DVD Drive, 500Gb Solid State Drive (from an old Retina Mac book and a GT1033 Video card and plugged it all together. I had read that the northbridge CPOU gets hot so I added a fan to the heatsink on the Northbridge.

The S7012 booted after a 30 seconds self-test, I installed Windows 10 and to my surprise, everything was detected.

After the first boot, the Windows 10 Task Masnger reported 12 cores/24 threads and 48GB of memory.

On a cool winters night, the CPU’s were barely 10c above ambiet temperatures.

Time to run Cinebench R23 and stress every core.

Now CPU cores were hitting 39c under 100% load.

Cinebench R23 was reporting 8277 points, that’s only 20% behind my AMD Ryzen 3700X

What was concerning was 366 Watts full load (all Core stress test

Power Savings

I set the Windows Power plan to “Power Savings” and now the power dray is a respectable 128W.

Using more than 4x PCIe Cards

In order to use more than 4 PCI cards, I had to buy some PCI over USB extension cards. The Heatsinks were blocking 2x PCIe slots.

In routed 2x PCI-E slots over USB to allow the Heatsink to intake air.

I routed 1x sound cards and 1x USB 3 card to the vertical PCIe card mounts on the case.

Looks ghetto but it works

Filling the empty memory banks

I managed to find 6 more RDIMM memory sticks that matched the memory that was installed. It was even made in the same week.

18 Memory banks were populated.

72GB memory installed at full speed.

Fans

I installed the fans ready for summer.

I routed some case RGB under my power cable.

Update July 2023

I swapped cases and added a Radon 570 to this build (in PCIe Slot 3, 4th down from the CPU)

I need to mount this better as this was just a test.

I was planning on using a Bifufation and Power injector to add the extra 50W of power that PCIe 2 does not provide but the card worked without it.

I cannot mount this inside the PC as I added heatsinks to the UC’s on the board.

Performance

Before (Heaven benchmark running on a 1030 Ti running at 1920×1200)

After (Radeon 570)

Stats

HWINFO

Task Manager

Mounted the Video Card Inside

I moved the video card up a few slots and mounted it inside the case in PCIe 5 slot

Motherboard Layout

The WiFi card was moved up to the top PCIe slot via a USB Extender

RTX2080 8GB

I did not stop there, I has a RTX 2080 where it has a AIO water cooling loop die.

I replaced the thermal pads and purchased a heat sink from Ali Express.

I mounted the tested the Temperatures with Furmark as I was not happy with the 1mm thermal pads on the memory. The memory needs 0.75mm pads.

269FPS at 64c on a PCIe 16x video card running at 8x is nice. I am using this system on a 25” 240Hz Monitor. If I can get 0.75mm thermal pads I might be able to lower temps more? The crazy thing is the Tyan S7012 can supply 45W on the PCIe slot. No Hardware adapters or PCIe power injectors were needed.

Conclusion

Even though this system is 12 years old it boots Windows in 20 seconds (after self-test) and it is a fast machine to use.

Would I game on this? Yes now that I added a Radeon 570

Would I code on it? Yes.

Would I run it 24/7? No.

Would I do this again? Yes

I still have a spare PCI-E slot for a SATA 3 card.

Handy Links

Wemos Mini D1 Pro Pinout Guide

March 6, 2022 by Simon

Here is my guide (v1.2) for the ESP8266 Wemos D1 Mini Pro pin’s.

Full Size Image: https://fearby.com/WeMosMiniD1ProGuide.png

Diagnosing a Windows 10 PC that will not post

August 7, 2021 by Simon

I had a friend who said his Windows PC was not booting and needed advice on how to fix it.

This post is to help others and to assist with a Warranty RMA ticket (spoiler one part died).

My Experience

I have been in the computer industry for 27 years working in the following areas.

Jobs I have worked in:

Computer Sales (Pentium 1 and Cyrix Media GX days)
Software Engineer (medical software and Y2K prep work)
Computer Repairs and Installation
IT Helpdesk
Web Content Administrator
Software Developer (Data Design and Development)
System Administrator

In this time I have built possibly 50 computers and reinstalled the following versions of Windows thousands of times

MS-DOS 6.22
Windows 3.11
Windows NT 3.5
Windows 95
Windows NT 4
Windows 98/98SE
Windows 2000
Windows Me
Windows XP
Windows Vista
Windows 7
Windows 8/8.1
Windows 10 (v1507, v1511, v1607, v1703, v1709, v1803, v1809, v1903, v1909, v2004, v20H2, v20H1, v20H2)
etc Windows Server

How hard can it be, it was possibly a dead power supply or corrupt BIOS based on what I knew.

The (Dead) System Specs

The specifications of the PC are as follows.

Mainboard: ASUS TUF AM4 B450-PLUS Gaming (v1.03)
Processor: AMD Ryzen 3600X (6 core 12 threads)
Graphics Card: ASUS TUX 1660Ti 6GB Memory
Memory Corsair Vengenace LPX 16GB 3000Mhz DDR4 (2x sticks)
Silverstone: 550 Power Supply (SST-ET550-B V1.1)
Storage 1: 500GB M.2 nVME SSD
Storage 2: 2TB Magnetic Hard Drive
Case: Antec Dark Phantom DP301M Case
Operating System: Windows 10 Home

The PC was pre-built and tested by a leading Australian store on the 12th of December 2019 (#1225616)

Diagnostics Plan

This was my roughy plan

Warning: Do not open a computer without an anti-static strap and use a clean static-free earthed surface (do not walk on carpet and touch computer parts). Make sure the power cable is not connected to the power supply and let the power in the system dissipate over 30 minutes first.

Look for external damage or damage to the parts inside (as this can tell a story) and check connections).
Evaluate the dust inside the case and power supply. Dust is the prime candidate for PC’s shutting down.
Plugin the system and try and boot it (and smell evidence of “Blue Smoke” from a dead part)
Follow the evidence (check that all fans spinning, look in the BIOS and investigate windows errors).

0. BIOS Beep codes

The PC had no BIOS Speaker installed one

On power on the PC speaker made a short beep (video card present but no kleyboard_ code)

Nothing displayed on screen and I tried another keyboard but no luck.

1. Look for Damage and Check Connections

There was no external damage on the corners or the case or damage to parts inside of the PC.

All external case screws are connected and present. I opened the side panel (thumbscrews) and have a look around.

All internal cables were firmly connected and not loose.

ATX-12V/EPS 8 Pin CPU Power cord was well seated in the Mainboard
The 24 pin Power plug was well seated.
The CPU Fan power plug and 21x Case fans were connected correctly.
The M.2 SSD and SATA Drive cable was connected.
The SATA drive had power.
All cables were routed cleanly and were zip-tied down.

2. Dust

My heart sank when I saw that there was little dust inside the PC. Dust is usually is a good sign that the PC can be quickly repaired as dust can block fans.

ATX PC INside, Mainboard, Video Card and memory

I checked the Antec Dark Phantom DP301M Case top, front panel and bottom dust filters and they were a bit dusty but not bad.

Looks like Dust is not an issue. I know the family who uses this PC are not smokers or cat owners so this explains the clean PC.

3. Start the System

I started the system with my nose right behind the back of the PC to quickly smell evidence of past or present electrical shorting.

When I pressed the Power button to turn on the Power Supply, CPU, Video card and case fans all spun up. No bad smell from dead parts. I kept tapping the delete key every few seconds to hopefully enter the BIOS. After a few seconds, the PC did not fire up the Video card image connected to a 1080P screen.

4. Follow the Evidence

The PC had no PC Speaker so I connected a spare one I had. The BIOS was not loading and no BIOS beep codes came out of the speaker.

Hmmm, Maybe the mainboard or video card is dead. I cannot get an image on a screen?

I shut down the PC and installed an MSI 1030 Ti video card and tried to start the PC.

No luck, the system would not boot with a working video card.

I installed known good DDR 4 memory into the system to test the memory.

Installing working memory did not help.

I plugged in a Radeon 570 Video card and it to would not boot.

I plugged in the ASUS TUF 1660Ti video card into another working PC and it worked just fine. The video card was not dead.

Maybe the Power supply is not properly working?

I plugged in a working 750W power supply to the system.

The external working power supply did not allow the PC to boot.

Maybe the Mainboard, Mainboard BIOS or processor is dead?

I read the manual and tried restoring the BIOS and reloading the BIOS with the ASUS Crash Free BIOS.

BIOS Recovery

The manual had a nice QR code link with instructions to follow.

I placed a new BIOS onto a USB key as instructed, renamed it and started the PC and waited 10 minutes.

The BIOS did not load.

Dead Processor?

In my 27 Years, I have only heard of one processor dying (an AMD Duron 600Mhz with no internal temperature sensors where the heatsink fell off). I was 99% certain that the Mainboard was dead. I gave the processor a 1% chance of being dead.

The only way to confirm this was to swap in an AMD Ryzen 3200G processor into the dead system.

I removed the heatsinks from the donor and dead systems and cleared the thermal paste off the processor(s) and heatsink(s)

When I removed the heatsink from the dead system there was some dust under the heatsink fins. There was a good coating of thermal compound on the heatsink.

I had plenty of fresh thermal compound so I used a fresh application on each processor installation.

Starting the dead system with a donor CPU

I started the dead PC with a known good AMD Ryzen 3200G processor and to my amazement the system booted and I was able to enter the BIOS.

I am glad I did not order a new mainboard on a 99% hunch as it looks like the AMD Ryzen 3600X is dead.

I reviewed BIOS settings (nothing unusually) and I loaded optimized defaults and booted the PC to Windows.

Windows loaded just fine.

After a few hours of testing, updating drivers and software updates I rebooted to the BIOS and applied the latest BIOS.

Maybe a fresh BIOS will get the system loaded with the Ryzen 3600X processor?

The new BIOS applied just fine.

Is the Ryzen 3600X dead?

I installed the Ryzen 3600X into a good PC, applied thermal paste and a heatsink (for an AMD 3700X) and tried to boot the PC, It would not boot.

Looks like the AMD 3600X is dead.

I returned the Ryzen 3200G to its PC and reinstalled the dead Ryzen 3600X to its PC, Maybe the new BIOS will get it going?

Conclusion

I will submit an RMA as this is still covered under warranty.

We received an RMA authorization

I checked every CPU pin and took a photo (10x photos)

The CPU Box did not have a plastic holder for the CPU fan so I wrapped it in bubble wrap.

I placed the CPU in its clam shell and an anti-static bag

CPU Box

Bubble Wrap

Packaged ready to send for RAM

Now we wait.

How to create a Private GitHub repository and access via SSH with TortiseGIT

July 17, 2021 by Simon

Here is a guide on using the GUI TortoiseGIT, GIT for Windows with GitHub.

Creating a GitHub Account

Make sure you have a GitHub account (free is OK) https://github.com/

I used my defined hardware Security Key

I clicked New Repo

Repo Settings

I set Private
Added a readme
Set .gitignore for Visual Studio files
I had no idea what licence to select (read more on licences here)

I clicked Create repository

Creating RSA Keys

Now I will create an SSH key to protect communications with GitHub.

When logged into GitHub, I clicked the SSH and GPG Keys menu

I clicked New SSH Key button

GitHub wants a public SSH Key

Setup Git Locally

I checked to see if the “git” command was installed on my local machine.

I will install Git from https://gitforwindows.org/

I download Git-2.32.0.2-64-bit.exe and started the installer

Folder C:\Program Files\Git

Set Options (e.g I like Check for Daily updates)

Start Menu Folder

Text Editor: I like Sublime Text from https://www.sublimetext.com/3

Set an Initial Branch name

Set 3rd party software options

Set HTTPS options

Set Line Endings preferences

Set Terminal Emulators preferences

Set Git Pull default preferences

Set Credential Manager preferences

Set Extra Options

Experimental Options

Installing

First Launch

Git is installed

Install Tortoise GIt

Visit https://tortoisegit.org/

Download Gif (64bit)

I started the installer.

Licence Agreement

SSH Type

Components

Install

Installing (Wizard)

Language: English

Shell Extension

I checked for a known git path.

I set my desired git username and email

I set desired SSH settings

Folder for Code Repositories

I create C:\Code\MyTestRepo

Before I can clone a repository I need o set the past to SSH in TortoiseGit.

I right-clicked on C:\Code\MyTestRepo and selected TortiseGit\Settings

I expanded the Network Node and set the following as the SSH Client

“C:\Program Files\TortoiseGit\bin\TortoiseGitPLink.exe”

I saved the Settings.

Clone a GitHub Repository via git and Tortoise GIT

Now I can clone a repositor to C:\Code\MyTestRepo,

I right clicked on C:\Code\MyTestRepo in explorer and clicked Git Clone

I added by GIT URL (Obtained from GitHub), I also set the path for the Private Key I created earlier.

I was informed that the key is in the wrong format

I need to convert the RAS Private Key to a Putty PPK Key

Convert an RSA Key to a Putty PPK with PuuttyGen

I opened the Open Putty Gen

I click Load and select the id_github_test private key I created earlier

I entered my private key password

The private key loaded.

Now I can export the private key to a PPK format

Clone (with a PPK Private Key)

Getting back to cloning a repository I right-clicked on C:\Code\MyTestRepo

I set the PPK private key

I entered the password

I accepted the key

The Repository started cloning.

Creating a Quick Visual Studio Project

I created a quick Visual Studio project and saved it to C:\Code\MyTestRepo\Hello World

Visual Studio files were visible in C:\Code\MyTestRepo

I Commit the new files to the repository (main branch)

I selected desired files to commit

Files were added

I right-clicked on C:\Code\MyTestRepo and selected TortoiseGit\Push

Files are now visible in GitHub

Awesome

Setup on Multiple Machines

I set up Git on multiple machines, cloned, made chances, committed those changes and pushed. I was able to Sync down those changes to each machine.

Good advice on Using Git

Happy Coding

Recovering a Dead Nginx, Mysql, PHP WordPress website

July 10, 2021 by Simon

(laptrinhx.com – do not steal this post)

In early 2021 www.fearby.com died and it was all my fault. Here is my breakdown of the events.

On the 4th of January 2021, I woke to see my website not loading.

Cloud flare reporting my websitre was unavailable.

https://www.fearby.com had 2 servers.

Web server (www.fearby.com)
Database server (db.fearby.com)

Upon investigating why my website was down I found out that WordPress could not talk to the database server. I tried to log into the database server via SSH failed (no response). I tried logging into my db.fearby.com server via the root console and it too did not work.

I was locked out of my own server and memory told me it was caused by my be playing with fail2ban and other system auditing tools a few months earlier.

I tried restoring the db.fearby.com serer from backups (one at a time). I had the last 7 days as individual backups. I had no luck, all of my backups were no good (I sat on the problem too long).

In mid-2020 I locked myself out of db.fearby.com (SSH and root console) because I setup an aggressive fail2ban, AIDE intrusion detection system(s) and firewall rules. I could no longer access my db.fearby.com server via SSH or the root console. The database server was still operational and I foolishly left it running (with no access).

I did not know how to (or had enough time) reset the root password on the Debian server. I was unable to think of a fix to restore my website. I should have reset the root password, it is easy to do thanks to a post from Janne Roustemaa – How to reset root password on cloud server.

A few months ago I finally found out how to reset the root password of a Debian server.

How to Reset the Root Password on a Debian server on UpCloud

I followed Janne Roustemaa’s guide here: How to reset root password on cloud server.

I logged into the Up Cloud Hub.

I shut down db.fearby.com from the UpCloud Dashboard.

I created a backup of db.fearby.com (just in case). I upgraded my backup plan from 1 backup every 7 days to daily backups for 7 days and weekly backups for 1 month.

Deploy a temporary server to reset the root password

I deployed a new temporary (cheap) server alongside the dead server in Chicago.

Get $25 free credit on UpCloud and deploy your own server: Use this link to get $25 credit (new UpCloud users only).

I called the server “recovery.fearby.com” and set Debian 9 and the Operation System (same as the dead server).

I added the command “shutdown -h 1” to the Initialization script to ensure the server shuts down after it was deployed.

I can only add the disk to the new server if the old db.fearby.com server is shut down.

I shut down db.fearby.com and recovery.fearby.com servers.

Both servers have shut down

Detach the disk from db.fearby.com

I detach the disk from db.fearby.com server.

In the UpCloud Dashboard, I opened the db.fearby.com and clicked the resize tab

I clicked the Detach button.

I clicked Continue

Attach db.fearby.com disk to recovery.fearby.com

Now I attached this disk as a secondary disk onto the recovery.fearby.com server.

In the UpCloud hub I clicked Servers then selected the recovery.fearby.com server, then clicked the Resize Tab

I scrolled down and clicked Attach existing storage

Attach existing storage dialogue

I selected the system disk from the db.fearby.com (that I detached earlier)

I clicked Add a storage device button

Now I have attached the storage from db.fearby.com and attached it to recovery.fearby.com as a secondary disk.

Starting the recovery.fearby.com server

I started the recovery.fearby.com server by clicking Start

The server is starting

When the server started, I obtained its IP and connect to it with MobaXTerm.

Now I can access the db.fearby.com disk.

I do not want to reset the root password until I undelete the files I need.

Viewing Disks

I ran this command to verify the attached disks.

lsblk

Two disks were visible

Alternatively, I can view partitions with the following command

cat /proc/partitions
major minor  #blocks  name
 254        0   26214400 vda
 254        1   26213376 vda1
 254       16   52428800 vdb
 254       17   52427776 vdb1

I can see partition data with these commands

recovery.fearby.com disk: /dev/vda1

fdisk -l /dev/vda1
Disk /dev/vda1: 25 GiB, 26842497024 bytes, 52426752 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

db.fearby.com disk: /dev/vdb1

fdisk -l /dev/vdb1
Disk /dev/vdb1: 50 GiB, 53686042624 bytes, 104855552 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

I ran this command to mount the second disk

mount /dev/vdb1 /mnt

I looked in the “/mnt/Backup” folder as this was where I had daily MySQL dumps saving too on the old system.

The folder was empty. It looks like the file system was corrupted.

Finding Deleted Files

I assumed the file system was corrupt, I installed Testdisk as I wanted to recover deleted SQL dump backups in the backup folder.

sudo apt-get update
sudo apt-get install testdisk

I confirmed testdisk was installed

photorec --version

I ran testdisk and passed in the disk as a parameter

sudo photorec /dev/vdb1

I selected the Disk /dev/vdb1 – 53GB and pressed enter

I selected ext4 partition and pressed enter (not whole disk)

I selected ext2/ext3/ext4 filesystem and pressed enter

I selected Free to only scan in unallocated space and pressed enter

When asked to choose the recovery location to restore files to I selected /recovery (on the recovery.fearby.com disk , not the db.fearby.com disk.

I just realized that the recovery.fearby.com disk is 25Gb and the db.fearby.com disk is 50GB, lets hope I do not have more than 25GB of deleted files (or I will have to delete recovery.fearby.com and deploy a 100GB system) and run undelete again.

After 20 minutes 32,809 files were recovered to /recovery/recup_dir

I pressed CTRL+C to exit photorec

I changed directory to /recovery

cd /recovery

I counted the recovered files

find . -type f | wc -l
>32810

Recovered Files were placed in sub folders.

drwxr-xr-x 68 root root  4096 Jan 19 13:28 .
drwxr-xr-x 23 root root  4096 Jan 19 13:28 ..
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.1
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.10
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.11
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.12
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.13
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.14
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.15
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.16
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.17
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.18
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.19
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.2
drwxr-xr-x  2 root root 36864 Jan 19 13:22 recup_dir.20
drwxr-xr-x  2 root root 24576 Jan 19 13:22 recup_dir.21
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.22
drwxr-xr-x  2 root root 24576 Jan 19 13:22 recup_dir.23
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.24
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.25
drwxr-xr-x  2 root root 32768 Jan 19 13:22 recup_dir.26
drwxr-xr-x  2 root root 32768 Jan 19 13:22 recup_dir.27
drwxr-xr-x  2 root root 36864 Jan 19 13:22 recup_dir.28
drwxr-xr-x  2 root root 32768 Jan 19 13:22 recup_dir.29
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.3
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.30
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.31
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.32
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.33
drwxr-xr-x  2 root root 36864 Jan 19 13:22 recup_dir.34
drwxr-xr-x  2 root root 36864 Jan 19 13:22 recup_dir.35
drwxr-xr-x  2 root root 36864 Jan 19 13:22 recup_dir.36
drwxr-xr-x  2 root root 32768 Jan 19 13:22 recup_dir.37
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.38
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.39
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.4
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.40
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.41
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.42
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.43
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.44
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.45
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.46
drwxr-xr-x  2 root root 20480 Jan 19 13:24 recup_dir.47
drwxr-xr-x  2 root root 20480 Jan 19 13:24 recup_dir.48
drwxr-xr-x  2 root root 20480 Jan 19 13:24 recup_dir.49
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.5
drwxr-xr-x  2 root root 20480 Jan 19 13:24 recup_dir.50
drwxr-xr-x  2 root root 20480 Jan 19 13:24 recup_dir.51
drwxr-xr-x  2 root root 20480 Jan 19 13:25 recup_dir.52
drwxr-xr-x  2 root root 20480 Jan 19 13:25 recup_dir.53
drwxr-xr-x  2 root root 20480 Jan 19 13:25 recup_dir.54
drwxr-xr-x  2 root root 20480 Jan 19 13:25 recup_dir.55
drwxr-xr-x  2 root root 20480 Jan 19 13:26 recup_dir.56
drwxr-xr-x  2 root root 20480 Jan 19 13:26 recup_dir.57
drwxr-xr-x  2 root root 24576 Jan 19 13:26 recup_dir.58
drwxr-xr-x  2 root root 36864 Jan 19 13:26 recup_dir.59
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.6
drwxr-xr-x  2 root root 20480 Jan 19 13:26 recup_dir.60
drwxr-xr-x  2 root root 20480 Jan 19 13:26 recup_dir.61
drwxr-xr-x  2 root root 20480 Jan 19 13:27 recup_dir.62
drwxr-xr-x  2 root root 20480 Jan 19 13:27 recup_dir.63
drwxr-xr-x  2 root root 20480 Jan 19 13:27 recup_dir.64
drwxr-xr-x  2 root root 20480 Jan 19 13:28 recup_dir.65
drwxr-xr-x  2 root root 12288 Jan 19 13:28 recup_dir.66
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.7
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.8
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.9
66 folders were recovered

I calculated the folder size

sudo  du -sh /recovery
13G     /recovery

I immediately started downloading ALL recovered files (13GB) to my local PC with MobaXTerm

On recovery.fearby.com I searched for any trace of recovered *.sql files that I was dumping daily via cron jobs

find /recovery -name "*.sql"

Many pages of recovered sql files were listed

/recovery/recup_dir.49/f36487168.sql
/recovery/recup_dir.49/f35110912.sql
/recovery/recup_dir.49/f36667392.sql
/recovery/recup_dir.49/f36995072.sql
/recovery/recup_dir.49/f35667968.sql
/recovery/recup_dir.49/f35078144.sql
/recovery/recup_dir.49/f37535744.sql
/recovery/recup_dir.49/f36913152.sql
/recovery/recup_dir.49/f33996800.sql
/recovery/recup_dir.49/f36421632.sql
/recovery/recup_dir.49/f35061760.sql
/recovery/recup_dir.49/f36143104.sql
/recovery/recup_dir.49/f36618240.sql
/recovery/recup_dir.49/f34979840.sql
/recovery/recup_dir.49/f37273600.sql
/recovery/recup_dir.49/f35995648.sql
/recovery/recup_dir.49/f36241408.sql
/recovery/recup_dir.49/f37732360.sql
/recovery/recup_dir.49/f34603008.sql
/recovery/recup_dir.49/f33980416.sql
/recovery/recup_dir.1/f0452896.sql
/recovery/recup_dir.1/f0437184.sql
/recovery/recup_dir.1/f0211232.sql
/recovery/recup_dir.17/f16547840.sql
/recovery/recup_dir.17/f16252928.sql
/recovery/recup_dir.17/f15122432.sql
/recovery/recup_dir.17/f17159720.sql
/recovery/recup_dir.17/f15089664.sql
/recovery/recup_dir.17/f15958016.sql
/recovery/recup_dir.17/f15761408.sql
/recovery/recup_dir.57/f69582848.sql
/recovery/recup_dir.57/f69533696.sql
/recovery/recup_dir.57/f69173248.sql
/recovery/recup_dir.57/f68321280.sql
/recovery/recup_dir.57/f70483968.sql
/recovery/recup_dir.57/f70746112.sql
/recovery/recup_dir.57/f68730880.sql
/recovery/recup_dir.57/f67862528.sql
/recovery/recup_dir.57/f70123520.sql
/recovery/recup_dir.57/f68337664.sql
/recovery/recup_dir.57/f70172672.sql
/recovery/recup_dir.57/f71057408.sql
/recovery/recup_dir.57/f68796416.sql
/recovery/recup_dir.57/f70533120.sql
/recovery/recup_dir.57/f69419008.sql
/recovery/recup_dir.57/f68239360.sql
/recovery/recup_dir.57/f69779456.sql
/recovery/recup_dir.57/f68255744.sql
/recovery/recup_dir.57/f67764224.sql
/recovery/recup_dir.57/f71204864.sql
/recovery/recup_dir.57/f70336512.sql
/recovery/recup_dir.57/f68501504.sql
/recovery/recup_dir.57/f67944448.sql
/recovery/recup_dir.50/f39059456.sql
/recovery/recup_dir.50/f38518784.sql
/recovery/recup_dir.50/f40206336.sql
/recovery/recup_dir.50/f40927232.sql
/recovery/recup_dir.50/f39485440.sql
/recovery/recup_dir.50/f39092224.sql
/recovery/recup_dir.50/f40861696.sql
/recovery/recup_dir.50/f39731200.sql
/recovery/recup_dir.50/f40337408.sql
/recovery/recup_dir.50/f38862848.sql
/recovery/recup_dir.50/f41664512.sql
/recovery/recup_dir.50/f41074688.sql
/recovery/recup_dir.50/f40828928.sql
/recovery/recup_dir.50/f41713664.sql
/recovery/recup_dir.50/f38092800.sql
/recovery/recup_dir.50/f39878656.sql
/recovery/recup_dir.50/f38305792.sql
/recovery/recup_dir.50/f38830080.sql
/recovery/recup_dir.50/f39534592.sql
/recovery/recup_dir.50/f39813120.sql
/recovery/recup_dir.50/f40435712.sql
/recovery/recup_dir.50/f41467904.sql
/recovery/recup_dir.50/f37901728.sql
/recovery/recup_dir.50/f38682624.sql
/recovery/recup_dir.50/f38191104.sql
/recovery/recup_dir.50/f38174720.sql
/recovery/recup_dir.50/f40878080.sql
//and many more

It would take a few hours to download 32,000 files from the other side of the world. Next time I deploy fearby.com, I will deploy it to Sydney Australia.

It looks like MobaXTerm does not copy to the selected destination that I selected when dragging and dropping files, Being impatient I scanned my system for a file that MobaXTerm had copied. MobaXTerm saves download to “%Documents%\MobaXterm\splash\tmp\dragdrop\”.

I opened some of the recovered SQL files and it looks like all files were partial and were not the whole database backup. A compete mysql dump should be over 200 lines long. Dang.

I downloaded all files I could from the these folders

/mnt/etc/nginx
/mnt/var/lib/mysql
/mnt/Scripts

Disappointed, I sat on things for a few weeks, I thought I had lost my website.

Try 2 – Reinstall a fresh db.fearby.com

After I copied files from the old db.fearby.com serve to recovery.fearby.com, I reattached the storage to the old db.fearby.com system.

In vein, I tried mending the broken MySQL service on db.fearby.com. I tried uninstalling and reinstalling MySql on db.fearby.com (each time no luck). I was having trouble with MySQL not starting.

I had too many rabbit holes (mysql errors) to list. I thought my database was corrupt.

I was kicking myself for letting my access to db.fearby.com lapse, I was kicking myself for not having more backups.

Try 3 – Reinstall MySQL

I tried deploying a new server and setting it up, maybe from the frustration I did not do it correctly. I had HTTPS issues with CloudFlare. I gave up for a few months

Try 4 – Check for Database Corruption

I downloaded DiskInternals – MySQL Recovery and scanned my database. To my amazement, it reported no database corruption.

Maybe I can recover my website?

Try 5 – Using RunCloud.io to deploy a website

Having failed with setting up a server from scratch, a friend (Hi Zach) said I should try RunCloud to deploy a server.

I tried to document everything from attaching RunCloud to UpCloud and Cloudflare’s API to deploying a server.

Long story short, RunCloud is not for me. I had too many issues with RunCloud and IPV6, CloudFlare API Integration, No SSH access to my server, no Nginx editing capabilities with RunCloud.

I deleted the RunCloud deployed server.

Try 6 – 10 minutes deploying a serer by hand

I ended up deploying a server in 10 minutes manually without taking notes.

Summary

I deployed a server (this time to Sydney).

I Installed the UFW firewall (configured and started)

sudo apt-get install ntp

Set the date and time

sudo timedatectl set-timezone Australia/Sydney

Installed ntp time server

sudo apt-get install ntp

I Installed PHP 7.4 and PHP 7.4 FPM (I cheated and googled: https://www.cloudbooklet.com/install-php-7-4-on-debian-10/ )

I edited PHP Config

sudo nano /etc/php/7.4/fpm/php.ini

Installed NGINX webserver

sudo apt-get install nginx

Configured NGINX (I got an CLoudflare to NGINX SSL Certificate), I also sighned up for a Cloud flare SSL certificate

sudo nano /etc/nginx/nginx.conf

Contents

worker_cpu_affinity auto;
worker_rlimit_nofile 100000;
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 768;
         multi_accept on;
}

http {
        root /www-root;
        client_max_body_size 10M;

        proxy_connect_timeout 1200s;
        proxy_send_timeout 1200s;
        proxy_read_timeout 1200s;
        fastcgi_send_timeout 1200s;
        fastcgi_read_timeout 1200s;

        ##
        # Basic Settings
        ##
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;

        # server_tokens off;
        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1.2 TLSv1.3;
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        ##
        # Gzip Settings
        ##


        gzip on;
        gzip_disable "msie6";

        gzip_vary on;
        gzip_proxied any;
        gzip_comp_level 6;
        gzip_buffers 16 8k;
        gzip_http_version 1.1;

        gzip_types text/plain application/xml;
        gzip_min_length 256;
        gzip_proxied no-cache no-store private expired auth;


        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}

I edited sudo nano /etc/nginx/sites-available/default

Contents

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        # SSL configuration
        #
        listen 443 ssl default_server;
        listen [::]:443 ssl default_server;

        ssl on;
        ssl_certificate /path/to/ssl/certs/cert.pem;
        ssl_certificate_key /path/to/ssl/private/key.pem;

        root /path-to-www;

        # Add index.php to the list if you are using PHP
        
        index index.html index.php;

        server_name fearby.com;

        #Security Headers
        add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
        add_header X-Content-Type-Options "nosniff" always;
        add_header Referrer-Policy "no-referrer-when-downgrade";
        add_header X-XSS-Protection "1; mode=block" always;
        add_header X-Frame-Options SAMEORIGIN always;
        add_header Permissions-Policy "accelerometer=(), camera=(), geolocation=(), gyroscope=(), magnetometer=(), microphone=(), payment=(), usb=()";

        # Force HTTPS
        if ($scheme != "https") {
                return 301 https://$host$request_uri;
        }

        # DENY RULES
        location ~ /\.ht {
                deny all;
        }
        location ~ ^/\.user\.ini {
                deny all;
        }
        location ~ (\.ini) {
                return 403;
        }

        if ($http_referer ~* "laptrinhx.com") {
                return 404;
        }

        if ($http_referer ~* "bdev.dev") {
                return 404;
        }

        if ($http_referer ~* "raoxyz.com") {
                return 404;
        }

        if ($http_referer ~* "congtyaz.com") {
                return 404;
        }


        location / {
                try_files $uri $uri/ /index.php?$args;
        }

        location ~ \.php$ {
                include snippets/fastcgi-php.conf;
                fastcgi_pass unix:/run/php/php7.4-fpm.sock;
        }

        # DNS
        resolver 1.1.1.1 1.0.0.1 valid=60s;
        resolver_timeout 1m;
}

I tested Nginx and PHP.

I installed MySQL (I Googled a guide)

I created a database, database user and assigned permissions for my blog.

I installed the WordPress CLI tool

I installed WordPress the using the wp-cli tool

wp core install --url=example.com --title=Example --admin_user=supervisor --admin_password=strongpassword [email protected]

When I had a blank WordPress I uploaded the blog folder that I backed up before.

I ran this command to allow Nginx to read the backed up website files

sudo chown -R www-data:www-data /path-to-www

I also uploaded the backup of my mysql database to /var/lib/mysql/oldblogdatabase

I ran this command to allow mysql to read the backed up database

sudo chown -R mysql:mysql /var/lib/mysql/oldblogdatabase

I also uploaded the following files to /var/lib/mysql

TIP: These files are very important to restore. You cannot just copy a database in a subfolder.

ibdata1
ib_logfile0
ib_logfile1
ib_logfile1
ibtmp1

I ran this command to allow mysql ro read the ib* files

sudo chown -R mysql:mysql /var/lib/mysql/ib*

I was able to load my old website

I still have some issues to solve but it is back.

Lessons Learned

Save all passwords and have backup accounts and roll back before working backups are gone.
Setup a Dev Test, Pre Prod Environment and do no test on production servers.
Do not delay disaster recovery actions.
Do not rely on automation.
Have more than a weeks worth of backups.

Get $25 free credit on UpCloud and deploy your own server: Use this link to get $25 credit (new UpCloud users only).

Change Log

Version 1.2

laptrinhx.com is stealing website content

December 23, 2020 by Simon

I searched google for one of my articles and I noticed that the website https://laptrinhx.com had copied over a hundred of my blog posts and were re-publishing them at https://laptrinhx.com/tag/fearby-com

I could see they copied 127 of my blog posts. They never asked for my permission.

laptrinhx.com is copying the text content and were loading resources from my website.

A quick browse revealed that they were copying hundreds of other peoples websites and passing it off as guest posts.

It looks like they are well known for stealing peoples content

Who are https://laptrinhx.com?

I could not see an about page or contact form but I can see these websites in the footer

https://laptrinhx.com/
https://bdev.dev/
https://raoxyz.com/
https://congtyaz.com/

It looks like Cloudflare is their Registrar.

Reporting Abuse to Cloudflare

I opened the Cloudflare Report Abuse Page

https://www.cloudflare.com/abuse/form

I submitted a report with the URL’s

Let’s see how long until Cloudflare takes down the pages.

Reporting the issue to the website owner

I also visited Cloudflare’s contact form (https://domaincontact.cloudflareregistrar.com/laptrinhx.com) for the site owner and requested they take down my content.

At first Cloudflare’s form failed to submit.

After a few retries I was able to submit this.

I was not happy with Cloudflare’s terse message about the site owner is under no obligation to reply.

If Cloudflare does not resolve this I am under no obligation to pay Cloudflare for services for my website.

Blocking Traffic from their domains

I edited my /etc/nginx/sites-available/default file and added the following rules to block requests from their servers.


if ($http_referer ~* "laptrinhx.com") {
        return 404;
}

if ($http_referer ~* "bdev.dev") {
        return 404;
}

if ($http_referer ~* "raoxyz.com") {
        return 404;
}

if ($http_referer ~* "congtyaz.com") {
        return 404;
}

I loaded the config and restarted NGINX

sudo nginx -t
sudo nginx -s reload
sudo service nixstatsagent restart

I checked the status of NGINX to ensure it came back up.

sudo systemctl status nginx

I will keep adding refer blocks if they are using other domains to scrape my site.

FYI: I resolved a CDN issue and also asked my CDN provider (EWWW Image Optimizer) to block image loads from these domains. EWWW.io are awesome.

I will keep updating this post with news from Cloudflare etc.

Good luck content owners.

Update: July 2021, Still no reply from Cloudflare, I submitted a second Copyright form with Cloudflare

Version: 1.5 Fixed image Links

How to perform a clean Windows 10 installation

November 28, 2020 by Simon

Twenty years ago it was quite common for me to reinstall Windows (Windows 3.11, 95/98/Me/2k/XP/Vista/7/8) every 6 months due to Windows becoming slow or error-prone. Luckily Windows 10 only needs to be reinstalled every year or two.

Fyi: Windows 10 has had 8 minor releases since 2018 and Windows 10 (Version 20H2) is the latest version.

In the last 3 months, my Windows (v2004) installation has become unresponsive and some USB devices are not detecting properly. I looked at my hard drive SMART data on my Windows drive and it has reported that Windows has been on for 5,152 hours, started 589 times and had 65 unsafe shutdowns (from Windows lockups).

Over 60 unsafe Widnbows shutdowns are not surprising as my Mainboard has some BIOS stability issues for 6 months.

I could have restored an Acronis True Image backup of my Windows installation but I did not keep a copy of a stable Windows from a few months ago. Lesson learned I have ordered larger hard drive to store more Acronis backups.

I can confirm Windows needs a reinstall by using a free program called DPC Latency Checker.

It shows frequent delays in Windows performing low-level tasks (red spikes in latency).

Running a system benchmark tool like Cinebench shows a 10% drop in system speed since the first time I ran this after building my system.

Backup, backup and backup

I always make sure I have backups of my files before reinstalling Windows.

I use Backblaze and Back Blaze Buckets to backup files and I use Acronis True Image software to backup my Windows 10 Partition. Every month I back up my Windows drive with Acronis and upload it offsite.

It’s time I reinstall Windows.

The Plan

I will perform a full (clean) Windows 10 installation by.

Backing up all important files.
Documenting all Hardware Drivers, Software and Configuration (drives and redirected folders).
Downloading the latest version of all of my driver and software.
Writing down all software serial numbers to re-activate Windows and software.
Creating a Windows 10 Install Image (USB Drive).
Shutting down and disconnecting all drives that are not C:\ Drive (e.g M.2 and SATA).
Installing Windows 10
Reinstall Drivers
Reinstall Essential Software
Setup Drive letters and redirect Special Folders.
Swap out a 2T drive for a larger 10TB Drive
Install BackBlaze and Inheriting my old PC backup state
Creating an Acronis True Image Backup of C:\ Drive (Windows)
Reinstalling Games
Drive Stats.
Misc

Background

I have a 1TB C Drive (M.2 NVME SSD) with Windows on it and a multiple spinning magnetic hard drives where I redirect Windows folders to.

Backing up all important files.

I am fortunate that I have redirected (moved) many of my system folders (e.g Favorites, Downloads, My Music, My Pictures, My Videos etc) to other drives and I can disconnect them to keep files safe before reinstalling Windows.

Never perform a Windows reinstall if you have files or data partitions on the same disk that Windows is installed on.

WinDirStat (View Files)

I will use a free program called WinDirtStat to find files still on C drive.

After scanning C Drive with WinDirStat I can spot large files and file types on my C Drive.

I can see a large Outlook file that needs backing up.

I moved all files I wanted to keep from C Drive to another Drive (that will be disconnected before reinstalling Windows).

I manually reviewed C:\ Drive for missed files to ensure I have backed up everything.

If I forget a file I can log in to my Backblaze backup history and restore it. I pay and extra $2 a month to ensure every file is kept for a year (all versions of every file too).

I backed up my Simon profile (default Windows User) folder with the free FreeFileSync Program.

All files I want to keep are off C:\ Drive.

Documenting all Hardware Drivers, Software and Configuration

I need to audit the location of all system folders so I can remap them after I reinstall.

I opened regedit.exe and navigate to the following registry path

a) Auditing Redirected Special Folders

HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders

In the past, I have redirected the following paths and I do not need to back these up as they are on other drives.

My Music = M:\Music
My Pictures = U:\Pictures
My Video = U:\Videos
Personal = U:\Documents

b) Auditing Drive Letters

I open Computer Management (Start, Run then compmgmt.msc) tool, click Disk Management and view attached Disks.

I will set up the same drive letters after the rebuild so BackBlaze backup can resume backup states. If I do not record the drive letters the new Windows will assign new drive letters. If I have new drive letters Back Blaze will re-upload 7TB of data. If I reassign the same drive letters post Install Back blaze will not get confused.

c) Auditing Software

I used SUMo (Software Update Monitor) software (read my review here) from https://www.kcsoftwares.com/?sumo to list all of my installed applications.

I exported all applications to a Spreadsheet in SUMo

d) Auditing Drivers

I used DUMo (Driver Update Monitor) also from https://www.kcsoftwares.com/?dumo to document my drivers.

I exported a list (Text file and Spreadsheet) in case I needed it later.

Now I am confident I have documented my current system.

Downloading the latest version of all of my driver and software

I pre-downloaded all of my computers drivers and apps that I might need after reinstalling Windows.

There is nothing worse than reinstalling Windows and not having drivers for Network or WiFi Cards.

I methodically store all files needed for a reinstall on a separate hard drive.

I have all fresh drivers needed to reinstall all devices in my new Windows.

Writing down all software serial numbers to re-activate Windows and software.

I checked to see if I have all of my serial numbers and passwords needed to activate software.

Windows 10
Office 2019
1Password
Cyber Duck
Back Blaze
EaseUS Partition Master
Acronis True Image
Bit Defender Antivirus
Proton VPN
SUMO and DUMo
Fritzing
MobaXTerm

I obtained my Windows 10 Serial number with help from here.

Creating a Windows 10 Install Image (USB Drive).

I can create a free USB based Windows installation media instead of using a Windows 10 DVD by visiting https://www.microsoft.com/en-gb/software-download/windows10.

I clicked “Download tool now“.

I ran the MediaCreationTool20H2.exe

I accepted the terms by clicking Accept.

Getting things ready appeared again.

After a few minutes, I was able to select “Create installation media (USB Flash drive, DVD, or ISO file) for another PC“

I accepted the defaults and clicked Next

I selected my USB Drive to save the installation media to.

The correct USB drive was selected

The Media Creation tool was busy “Getting a few things ready“

Downloading took about 20 minutes

The USB key is ready to use to Install Windows 10.

It’s time to install Windows 10 🙂

Shutting down and disconnecting all drives that are not C:\ Drive

Now I will shut down my PC and physically disconnected all of my secondary SATA hard drives (where the data is that I want to keep).

Experience has taught me to only connect the desired C drive disk when installing Windows. If other drives are present Windows can get confused.

I will leave my G:\ Games M.2 Drive connected (as I deleted all partitions on it). The second M.2 drive has a heatsink on it with sticky thermal adhesive strip and I do not want to damage it.

Now I am ready to wipe my old Windows drive and install a fresh copy.

Installing Windows 10

I inserted the freshly made Windows Install USB device into my PC USB port and restarted my PC. I entered my BIOS (by pressing F2). I configured my PC boot drive from the USB Key (not the drive with Windows on it).

On the next reboot, the Windows 10 Install program loaded from the USB key.

I set the language and keyboard location clicked Next.

I clicked Install now.

I entered my previously purchased Windows 10 Serial number and clicked Next.

I reviewed the license terms and clicked Next.

I had the following options.

Upgrade: Install Windows and keep files, settings and applications.
Custom: Install Windows 10 only (advanced)

I will choose Custom (advanced) as it will be a clean install.

I selected my C:\ Drive (1TB) as the drive to install Windows to, not the second 500GB M.2 SSD (I will use for games later).

I deleted the existing Windows partition to delete the existing Windows 10 installation.

I clicked OK to the message about additional partitions being created.

fyi: This is what the storage disk will look like after Window’s is installed

Windows gave me a breakdown of the partitions. Windows will install to the 931GB partition.

I clicked Next.

Windows is installing 🙂

After 10 minutes Windows had installed enough to start asking questions.

I selected Australia (my country).

I selected the US keyboard layout.

Windows took a moment to install some more files.

I was asked is this is a Personal or Business installation.

Personal of course.

I entered my Microsoft Account. This appears to be unskippable like in previous versions of Windows.

I entered my MFA password to verify my account.

Windows asked me to create a PIN (not a password)

I setup a pin

I disabled all options giving away my privacy

I skipped saving files to One Drive.

Windows resumed installing

I entered my Product Key for Office

I clicked Not Now to disable setting up Cortana

Windows processed files for 2 more minutes.

Windows Rebooted and it started downloading plug and play devices.

Windows 10H2 was installed and I was logged into the desktop.

That took less than 30 minutes.

Reinstall Drivers

I shut down my PC and plugged in all hard drives again.

70% of my drivers were auto set up by Windows and I just had to install the odd (pre-downloaded) driver install files.

It only took 5 minutes to install all missing drivers.

Reinstall Essential Software

I went through my list of applications that I wanted to be reinstalled and installed them one by one.

I skipped installing the older apps that I no longer wanted.

Setup Drive letters and redirect Special Folders.

I opened the Windows 10 Disk Manager application and reset all drive letters and te redirected Special Folders to their old values (e.g Favorites, Downloads, My Music, My Pictures, My Videos etc)

Swap out a 2T drive for a larger 10TB Drive

I noticed that one of my 2TB Drives was reporting (S.M.A.R.T data) that is was not healthy.

I ordered a 10TB Western Digital Gold Drive) to swap out the 10-year-old WD Green Hard Drive. I can use the extra space to keep 6 months of weekly Acronis backups too.

The 10TB drive arrived.

I plugged in the new 10TB WD Gold hard drive and gave it the temporary Y:\ drive letter. The drive was healthy (0 power-on hours) and not a refurbished drive.

I performed a Full format (not quick) to test every sector, this format took over 12 hours,

I swapped the drive letters M: became X:, and Y: became M:\ drive.

I copied (with FreeFileSync) everything from the aging 2TB drive to the new 10TB drive.

Copied data frop, the 2TB Drive to the 10TB drive

Before disconnecting the old 2TB Drive I copied data to as another backup. It may die, but may not.

Copying large amount’s of data between SATA drives takes a long time.

Install BackBlaze and Inheriting my old PC backup state

I have about 4TB of data backed up to Back Blaze on my older Windows installation. I could delete that and re-upload 4TB but In will Inherit my backup state from my older Windows and only back up what I have moved between drives.

Snip from Backblaze – Inherit Backup State

NO IU need to inherit my old Back Blaze backup state so I don’t have to re upload 4TB of data.

The Inherit Backup State feature allows a new installation of the Backblaze software to inherit an existing license and backup under the same account. This will prevent the unnecessary re-upload of files that have previously been backed up. This is useful in a number of circumstances, such as the following examples:
Inherit Backup State – Backblaze Help

To inherit the backup state I logged into Back Blaze and downloaded Back Blaze (trial), I Installed it and logged in.

Back Blaze installed files.

Back Blaze took 10 minutes scanning my hard drives.

Back Blaze reported I had 7+ TB of data on all of my disks.

I only want to backup 4TB of files.

Back Blaze has not prompted me to inherit a backup state yet???

I selected my desired drives to backup.

I also excluded folders that I did not wan’t backed up.

I clicked Inherit Backup State. I logged in again.

I could see previous backups states 🙂

Upon selecting a previous backup state I had to enter the Encryption key (created at the time of the previous backup creation).

Inheriting the previous backup state takes a while.

Please wait dialog.

Success the past backup state was intertied.

I double checked that my excluded paths and folders were still being excluded.

I could see the files left to upload.

I only had 970GB of files to upload now after inheriting 4.8TB of previously backed up data.

This is a relief, I always feel safer when I have online backups..

Creating an Acronis True Image Backup of C:\ Drive (Windows)

I use Acronis True Image to take full backups of my C:\ (read my review here).

I opened Acronis True Image 2021 and clicked the Backup Tab to specify what to backup (by default Acronis will back up all disks and partitions).

I clicked my PC’s name (ZENigma) and then clicked Disks and partitions.

I selected my C:\ Drive partition (named MP600_1TB) and then presses OK.

I selected Browse to specify a local drive to back up to.

Acronis True Image can backup to the cloud but I prefer to do that myself with BackBlaze (it’s way cheaper).

I selected M:\_Acronis_Backups\ZENigma as the location to Backup C:\ Drive to. M:\ now has 5TB free.

I set a schedule to backup at 7PM on Friday’s.

I specified Version Chain as the backup type.

Read more on backup types here: Acronis True Image 2021 Help – Backup Schemes

I configures email alerts.

I set backups to be split into 100MB chunks. I find 100MB is the optimal size to upload to BackBlaze.

I configured backup validation frequency.

Now my Windows (C:\ partition) is ready to be backed up.

TIP: It is a good idea to disable any of your Antivirus programs before performing a backup of system files.

I disabled my Bitdefender Antivirus until my next Windows Reboot.

I started my Acronis true Image backup of C:\ Drive.

I could see the backup is being created in the M:\_Acronis_Backups\ZENigma folder.

Create a System Image

Although I have little faith in Windows System Restore I will create a System Restore Image just in case.

If I am feeling lazy I can use this restore point to restore my system at a future date.

I can choose from past restore points.

Documenting Drives

While I am here I will document my hard disk and estimate the replacement date (10 years).

One hard drive was 10 years old and S.M.A.R.T Diagnostics was recommending I replace the drive ASAP. This is how I knew to buy a bigger hard drive (above).

Reinstalling Games

I created a partition on the games M.2 SSD and downloaded games to that drive. This took days.

All games.

Drive Stats (Old 2TB M:\ Drive)

fyi only:

Here is a chart of all of my drives:

Hard Drive – Power On Count:

All of my Magnetic rives are very old.

Hard Drive – Power On Hours.

Hard Drive – Read Benchmark

Hard Drive – Write Benchmark

One thing to note is NVME M.2 SSD’s are way faster than SATA drives.

How do I restore an Acronis True Image Backup

Acronis has been running for 2 weeks now and a restore of Windows is quite easy. Under the Backups menu, I can click Recovery then restore the Windows partition.

I can restore individual partitions if needed.

I can just restore files if I wish form Acronis inside Windows.

I followed the prompts to create a Universal Restore boot media

I installed the Acronis Universal Restore bootable media builder program.

I created a USB based Bootable rescue media.

I created a rescue media bootable usb stick

I can use this USB key to boot my system and restore and files, partitions of weekly snapshots of Windows.

Misc

I redirected my Windows Temp Folder to my new M Drive to prevent my Windows Solid-state drive getting worn out.

Conclusion

Windows is reinstalled and is super snappy and had less junk installed.

Reinstalling is much easier with a second hard drive and patience backing everything up at before you start reinstalling.

DPC Latency checker showed I had no issues with latency.

1.0 Initial Draft

MobaXterm tabbed SSH client (etc) for Windows

July 7, 2020 by Simon

After posting my last blog post (Goodbye Dropbox, One Drive, iCloud and Hello Nextcloud private cloud on UpCloud) I received comments from readers as to why I used Putty/WinSCP and not MobaXterm. To be honest I had no idea MobaXterm existed.

I did a quick Google and checked out the pros and cons of Putty/WinSCP v MobaXTerm.

MobaXterm Highlights

MobaXterm is a lot faster at uploading and downloading files over SSH connections
MobaXterm can connect to practically anything (SSH, Telnet, RDP, RSH, Xdmcp, VNC, FTP, SFTP, Serial, Shell etc)
Multiple terminals can either be in tabs or split horizontally or vertically.
Edit remote files via SSH and SFTP – Pro feature
Supports macro controls (multi commands in single screen) – Pro feature
MobaXterm has many plugins
MobaXterm can be run from a portable USB drive

Nice

MobaXterm Website

I visited https://mobaxterm.mobatek.net/

https://mobaxterm.mobatek.net/ website screenshot

I had an option to download the Free Version or to purchase a Professional Version.

I will be reviewing the Pro version (you will see why later). I am reviewing Version 20.2 (Build 249). You can download the free version at https://mobaxterm.mobatek.net/download.html.

If you purchase the Pro version (with 12 months of updates) for $69 USD. Delivery is done by email within 24 hours after payment is received.

Installing MobaXterm

I downloaded the 1.5MB Installer (the link was emailed to me after purchase)

I opened the MobaXterm installer and entered my Pro serial number (emailed to me) then I clicked Next

Customise Options (during the install)

I clicked Customise MobaXTerm Professional settings button

The customizer options are impressive (Import and export settings, enable/disable features, generate a portable package etc)

I had an option to customise the banner or any server I connect to but I left the banner as the default banner.

I reviewed all options for the application. This is more in depth than Putty for sure.

Enable or disable just about everything.

I reviewed my default SSH settings. SSH Keep alive will be turned on for sure.

I had an option to add plugins to MobXTerm

I had the option to generate a portable of full installer

Now I can resume the setup of MobaXterm, I clicked Next

I accepted the licence agreement and clicked Next

I installed MobaXterm to my C:\ Drive

I clicked Install

MobaXterm was now installed

MobaXterm

I opened MobaXterm

Apologies in advance I setup Dark Mode in MobaXterm as soon as I opened the Configuration. All screenshots will be shown below in Dark Mode.

There are 12 colour themes to choose from.

Main Interface

The MobaXterm interface is more feature packed than Putty.

Menu Bar, Toolbar, Tabs and options are available

Windows prompted me to allowed access to MobaXTerm in my Windows firewall.

Time to create a connection to a server.

Adding a SSH Connection to a Server

I created a new SSH server connection to one of my existing servers.

I specified the IP address, username and protocol.
I set “SCP (enhanced speed)“

I specified the path to my private key (for the server)

I reviewed other terminal settings.

I changed the font to Courier New, Size 12 (I am sad). I also increased the initial size of the terminal (width and height)

I reviewed network settings (all good)

I added a shortcut to the server onto my desktop

When I opened the connection to the server I was prompted to enter the Passphrase I set on my Public Key (nice).

Then I was prompted to put in my 2FA One Time Password (OTP) in the main terminal windows and a second time in a popup form (for the connection that builds the folder structure on the left). This is how I configured my website. To get the OTP I need to insert my YubiKey and one the YubiCo Authenticator app and enter my keys password.

Some people say that having a passphrase on a public/private key and 2FA Authentication is a bit overkill but that is not everything I do.

I need to enter two OTP’s to connect to my site.

If you are fast you can use the same OTP in both windows (but they expire in 30 seconds)

Nice I was able to connect to my Server just like Putty. The cool thing is I can see files and folders without opening WinSCP.

This is way more convenient than using two programs.

I enabled the Remote Monitoring status bar in MobaXterm. Now I can see system resources on my server without running CLI commands.

Upload Speed to Singapore from Australia

I uploaded a 1GB file with to the same server from WinSCP and MobaXterm to simulate a restore.

I was gobsmacked, WinSCP could only upload a file at 342KB/s, MobaXterm could upload at 4.42MB/s.

I was uploading a 1GB file from my house to a server in Singapore (within 5 minutes of each other). I did set “SCP (enhanced speed)” when Installing MobaXterm.

MobaXTerm is 12x faster than WinSCP at uploading for me. This is well worth upgrading to a Professional licence.

Download Speed (from USA/East Coast) to Australia

I ran a practical download test (this time from Chicago). In the right-hand side of MobaXterm I zipped up my entire website folder (1.66GB)

zip -r /temp/www-22-Jun-2020.zip /www-folder

Then in the left-hand side of MobaXterm I downloaded the file, this is as simple as right-click then download.

I was presented with a directory tree specify where to save the file to.

I noticed the file was downloading at 16Mbps then 18Mbps then 22Mbps within seconds.

The download speed settled down to 26Mbps after a few minutes.

This is amazing for a download from the other side of the world (200ms ping and on a VPN, 19 traceroute server hops away).

I downloaded the same file from the same server with WinSCP to compare and MobaXTerm was 10x faster at downloading. WinSCP could only manage 2.57/MB/s

Plugins

I will look into plugins soon: https://mobaxterm.mobatek.net/plugins.html

Local Terminal

I can also start a local terminal (documentation here)

I can run Linux commands on my Windows machine via Cygwin

“ls -al” on Windows anyone

I noticed MobaXterm had found my WSL 2 Installation of Debian on my Windows 10 v2020.

I can now use MobaXterm to open my WSL Linux installations.

MobaXterm can easily connect to my local and online Linux servers with ease.

Multiple Exec

I used the Multiple Exec (similar to the multiple cursors in Sublime Text Editor) to control 2 serves at once and run identical commands.

Import Putty Sessions

I did not notice until later but you can import existing Putty connections.

Generate Public/Private Keys

Also, you can generate SSH Public and Private Keys from the Tools menu or the left-hand side (tab).

Pros

File/Folder and Terminal window side by site
13x faster file uploads over SSH
10x faster downloads over SSH
Integration with WSL Linux Images
Import Putty Settings
Local Terminals/Cygwin

Cons

It would be nice to only have to enter my 2FA OTP once when connecting to one of my serves.
The user interface is full-on, I am still learning it

Conclusion

Honestly having faster uploads and downloads over SSH is a dream come true. I live in Australia and we have terrible latency and I have better things to do than to watch slow uploads/downloads.

I look forward to investigating MobaXterm plugins in the near future as I feel I I have just scratched the surface.

Links

MobaXterm Documentation: https://mobaxterm.mobatek.net/documentation.html

v.1.1 Plugin info fix