Code, InfoSec and Server Stuff
Views are my own and not my employer's
This is how I setup Certification Authority Authorization (CAA) DNS record(s) to prevent https cert issue/misuse on my domain and 1 sub domain.
Read the full article here: https://fearby.com/article/setup-a-certification-authority-authorization-caa-dns-records-to-prevent-https-cert-issue-misuse/
This is how I replacing Google Analytics with Piwik/Matomo for a locally hosted privacy-focused open source analytics solution
Advertisement:
Aside
I have a number of guides on moving away from CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. PHP is my programming language of choice.
Now on with the post
Google Analytics
I will fully admit Google Analytics is good. I posted this a while ago on how you can set up Google Analytics on your site.
Google Analytics has some great charts and graphs. Simple to set up and easy to use.
My site traffic is growing and I would prefer to hold my own analytics on user data. Matomo is an analytics solution that stays on my server and not in the hands of Google.
Google Analytics can be Slow
Sometimes the Google Analytics server is slow (affecting the speed of my server). I blogged recently about speeding up a WordPress site here and GOogl Servers were not adding expiry headers on assets.
I did log a ticket with Google to fix this and the experience was terrible.
Support for Google Analytics is terrible
GT Metrix scores show poor delivery of tracking assets.
Privacy
After the Cambridge Analytica fiasco (that made me decide to delete facebook) sending analytics to Google is not a good idea.
I am not saying Google is evil but I want my site’s visitors tracking data to remain local.
Website Speed Benchmark before installing Matomo
I can load my site in 1.3 seconds at best, 1.5 seconds on average and 2.0 seconds at worst. My site is loading 11 assets.
Page Speed Scores
Y Slow Scores, Gogol Assets are reporting no expiry headers (slowing down scores)
Advertisement:
Google Analytics tracking assets are slow.
Optimizations to be made
Browser caching is not possible with Google Analytics.
Missing Expiry Headers (I can see a Google Tag Manager server is slowing down my servers benchmark score)
Why Mamoto (instead of Google Analytics)
I came across
Someone pointed out that @haveibeenpwned got a bunch of traction on Reddit today. With pretty much everything now either cached by @Cloudflare or served by @AzureFunctions, the first I know of a 28x traffic increase is no longer when something scales it’s when someone tells me 😎 pic.twitter.com/ifj7nQg3n4
— Troy Hunt (@troyhunt) November 5, 2018
Mamoto was mentioned
It’s an Open Source, self hostable, privacy friendly alternative to Google Analytics:https://t.co/NiK7A7uQAE
— Lukas Winkler (@lw1_at) November 5, 2018
I visited https://matomo.org/
Snip
> Take care of running Matomo yourself by installing it on your own server. There is no cost for Matomo itself but you need a server and update Matomo & your server regularly to keep it fast and secure. Need help? The Matomo team provides free help resources and paid support.
Source Code
Source code is available.
> Matomo is the leading open alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites, apps & the IoT and visualise this data and extract insights. Privacy is built-in. We love Pull Requests! https://matomo.org/
https://github.com/matomo-org/matomo
Installation Guide
I read the installation guide here https://matomo.org/docs/installation/
You can view the changelog here https://matomo.org/changelog/
Downloading Mamoto
I logged into my server via SSH and downloaded the 18MB download to the desired folder
1 2 | cd /www-root/matomo-folder/ wget https://builds.matomo.org/matomo.zip |
I unzipped the zip file
1 | unzip matomo.zip |
I loaded the URL where Matoto was installed (e.g “https://fearby.com/folder/subfolder/matomo/”)
I received this well-crafted error.
Raw Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | An error occurred Matomo couldn't write to some directories (running as user 'www-usr'). Advertisement: <script async src="//pagead2.googlesyndication.com/pagead/js/adsbygoogle.js"></script> <ins class="adsbygoogle" style="display: block;" data-ad-client="ca-pub-9241521190070921" data-ad-slot="8866281408" data-ad-format="auto"></ins> <script> (adsbygoogle = window.adsbygoogle || []).push({}); </script> Try to Execute the following commands on your server, to allow Write access on these directories: chown -R www-usr:www-usr /www-root/folder/subfolder/matomo chmod -R 0755 /www-root/folder/subfolder/matomo/tmp chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/assets/ chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/cache/ chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/logs/ chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/tcpdf/ chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/templates_c/ If this doesn't work, you can try to create the directories with your FTP software, and set the CHMOD to 0755 (or 0777 if 0755 is not enough). To do so with your FTP software, right click on the directories then click permissions. After applying the modifications, you can refresh the page. |
I refreshed the page after running the commands above on my site (via SSH)
A system check was performed. I installed when PHP 7.2.11 was the latest, PHP 7.2.12 or higher might be available. Follow my guide to update PHP on Ubuntu.
I had one Issue with Freetype not being installed.
I solved this error by installing freetype
1 | sudo apt-get install freetype* |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | Reading package lists... Done Building dependency tree Reading state information... Done Note, selecting 'freetype-tools' for glob 'freetype*' Note, selecting 'freetype2-demos' for glob 'freetype*' The following NEW packages will be installed: freetype2-demos 0 upgraded, 1 newly installed, 0 to remove and 66 not upgraded. Need to get 123 kB of archives. After this operation, 728 kB of additional disk space will be used. Get:1 http://us.archive.ubuntu.com/ubuntu bionic/universe amd64 freetype2-demos amd64 2.8.1-2ubuntu2 [123 kB] Fetched 123 kB in 0s (965 kB/s) Selecting previously unselected package freetype2-demos. (Reading database ... 122574 files and directories currently installed.) Preparing to unpack .../freetype2-demos_2.8.1-2ubuntu2_amd64.deb ... Unpacking freetype2-demos (2.8.1-2ubuntu2) ... Processing triggers for man-db (2.8.3-2) ... Setting up freetype2-demos (2.8.1-2ubuntu2) ... |
Then I installed “php-gd”
1 | sudo apt-get install php-gd |
Output:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: libapache2-mod-php7.2 php7.2-cli php7.2-common php7.2-curl php7.2-dev php7.2-fpm php7.2-gd php7.2-json php7.2-mbstring php7.2-mysql php7.2-opcache php7.2-readline php7.2-xml php7.2-zip Recommended packages: apache2 The following NEW packages will be installed: php-gd php7.2-gd The following packages will be upgraded: libapache2-mod-php7.2 php7.2-cli php7.2-common php7.2-curl php7.2-dev php7.2-fpm php7.2-json php7.2-mbstring php7.2-mysql php7.2-opcache php7.2-readline php7.2-xml php7.2-zip 13 upgraded, 2 newly installed, 0 to remove and 53 not upgraded. Need to get 33.2 kB/6621 kB of archives. After this operation, 150 kB of additional disk space will be used. Do you want to continue? [Y/n] Y Get:1 http://ppa.launchpad.net/ondrej/php/ubuntu bionic/main amd64 php7.2-gd amd64 7.2.11-4+ubuntu18.04.1+deb.sury.org+1 [27.1 kB] Get:2 http://ppa.launchpad.net/ondrej/php/ubuntu bionic/main amd64 php-gd all 2:7.2+68+ubuntu18.04.1+deb.sury.org+1 [6036 B] Fetched 33.2 kB in 0s (75.9 kB/s) Reading changelogs... Done (Reading database ... 122597 files and directories currently installed.) Preparing to unpack .../00-php7.2-mysql_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-mysql (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Preparing to unpack .../01-php7.2-opcache_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-opcache (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Preparing to unpack .../02-php7.2-json_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-json (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Preparing to unpack .../03-php7.2-readline_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-readline (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Preparing to unpack .../04-php7.2-mbstring_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-mbstring (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Preparing to unpack .../05-php7.2-curl_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-curl (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Preparing to unpack .../06-php7.2-zip_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-zip (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Preparing to unpack .../07-php7.2-fpm_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-fpm (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Preparing to unpack .../08-php7.2-xml_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-xml (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Preparing to unpack .../09-php7.2-dev_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-dev (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Preparing to unpack .../10-libapache2-mod-php7.2_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking libapache2-mod-php7.2 (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Preparing to unpack .../11-php7.2-cli_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-cli (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Preparing to unpack .../12-php7.2-common_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-common (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ... Selecting previously unselected package php7.2-gd. Preparing to unpack .../13-php7.2-gd_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ... Unpacking php7.2-gd (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Selecting previously unselected package php-gd. Preparing to unpack .../14-php-gd_2%3a7.2+68+ubuntu18.04.1+deb.sury.org+1_all.deb ... Unpacking php-gd (2:7.2+68+ubuntu18.04.1+deb.sury.org+1) ... Setting up php7.2-common (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Processing triggers for ureadahead (0.100.0-20) ... Setting up php7.2-curl (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Setting up php7.2-mbstring (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Setting up php7.2-readline (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Processing triggers for systemd (237-3ubuntu10.4) ... Processing triggers for man-db (2.8.3-2) ... Setting up php7.2-json (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Setting up php7.2-opcache (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Setting up php7.2-mysql (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Setting up php7.2-gd (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Creating config file /etc/php/7.2/mods-available/gd.ini with new version Setting up php7.2-xml (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Setting up php7.2-zip (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Setting up php7.2-cli (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Setting up php-gd (2:7.2+68+ubuntu18.04.1+deb.sury.org+1) ... Setting up libapache2-mod-php7.2 (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Warning: Could not load Apache 2.4 maintainer script helper. Setting up php7.2-dev (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... Setting up php7.2-fpm (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ... |
I refreshed the Matomo setup wizard page, Freetype is now installed 🙂
Advertisement:
Database Settings
For the life of me, I could not get Matomo to talk to a database on another server so I set it up on my localhost.
I used this guide to help in mysql CLI to create the database and users.
Commands in mysql to create a database and user and assign the user to the database. If you are not comfortable with MySql CLI you can use Adminder GUI.
1 2 3 | CREATE DATABASE tbdatabasename; GRANT ALL PRIVILEGES ON tbdatabasename.* TO 'databaseuser'@'localhost' IDENTIFIED BY '#####################################'; GRANT SELECT ON tbdatabasename.* TO 'databaseuser'@'localhost'; |
I used this PHP code to test connecting to the dedicated server before using the localhost
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | <?php $servername = "localhost"; $username = "databaseuser"; $password = "#################"; $dbname = "tbdatabasename"; // Create connection $conn = new mysqli($servername, $username, $password, $dbname); // Check connection if ($conn->connect_error) { die("Connection failed: " . $conn->connect_error); } else { echo "Connection Success"; } $conn->close(); ?> |
Database created ok
I created a Matomo user then I grabbed the javascript tracking ID code so I could paste this into WordPress.
I opened my WordPress theme settings and deleted the Google tracking tags and added the Matomo tracking code.
I added the Matomo tracking javascript in the head section.
The dashboard is up and collecting data.
Some reports are missing data so I will come back later.
After 1 week I could see data
Advertisement:
Securing Mamoto
I read this guide here to secure Matomo
Opt Out Tracking
I enabled Opt Out Tracking in the Mamoto settings and added the generated opt-out code to my front page and at the bottom or all existing articles.
I had to allow iframe tags on my site by adding this header in NGINX (previously I blocked iframes)
1 | add_header X-Frame-Options sameorigin |
Add Opt Out Tracking Code to WordPress.
I updated my privacy page and my GDPR notification bar. Now visitors will see a opt out of tracking on the front page and all article pages.
SMTP Settings
I added my GSuite mail server settings to enable sending of reports via email. I loaded my old guide here to get the GSuite SMTP settings.
I enabled force https on the Mamoto application (edited: config/config.ini.php file)
1 2 3 | [General] ... force_ssl = 1 |
Matomo Plugins (Marketplace)
I opened the System then Plugins section of Matomo to open the Marketplace
I installed these plugins
Updating PHP
Matomo Admin (Panel – Security/Diagnostics) section will report if your PHP gets out of date.
Hardening Advice
Matomo also recommended some php.ini file changes.
> open_basedir – open_basedir is disabled. When this is enabled, only files that are in the given directory/directories and their subdirectories can be read by PHP scripts. You should consider turning this on. Keep in mind that other web applications not written in PHP will not be restricted by this setting.
> upload_tmp_dir – upload_tmp_dir is disabled, or is set to a common world-writable directory. This typically allows other users on this server to access temporary copies of files uploaded via your PHP scripts. You should set upload_tmp_dir to a non-world-readable directory
This may break your WordPress so enable at your own risk. I might move Mamoto to a dedicated “analytics” subdomain then enable these options.
Advertisement:
Troubleshooting
I had to run this command when installing Device Pixel Ratio, Device Network Information, Bandwidth plugins
1 | php /www-root/path/matomo/console core:update |
Output:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | *** Update *** Database Upgrade Required Your Matomo database is out-of-date, and must be upgraded before you can continue. The following dimensions will be updated: log_visit.device_pixel_ratio. *** Note: this is a Dry Run *** ALTER TABLE `matomo_log_visit` ADD COLUMN `device_pixel_ratio` DECIMAL(5,2) DEFAULT NULL; *** End of Dry Run *** A database upgrade is required. Execute update? (y/N) y Starting the database upgrade process now. This may take a while, so please be patient. *** Update *** Database Upgrade Required Your Matomo database is out-of-date, and must be upgraded before you can continue. The following dimensions will be updated: log_visit.device_pixel_ratio. The database upgrade process may take a while, so please be patient. Executing ALTER TABLE `matomo_log_visit` ADD COLUMN `device_pixel_ratio` DECIMAL(5,2) DEFAULT NULL;... Done. [1 / 1] Matomo has been successfully updated! |
GTMetrix (After)
GT Metrix reports that my site is not slower (still 1.5 seconds)
I can see that some JavaScript is not being picked up by CDN.
Also 2 More files loading (when compared to Google Analytics)
Time to add the Mamoto files to my CDN.
Adding Matomo Resources to a CDN
I read this Matomo forum post.
I copied these 2 assets to my WordPress wp-content folder (my WordPress CDN ewww.io will then upload them to the CDN).
1 2 3 4 | cd /www-root/wp-content/ cp /www-root/utils/matomo/piwik.js ./piwik.js cp /www-root/utils/matomo/plugins/CoreAdminHome/javascripts/optOut.js ./optOut.js chown www-data:www-data *.js |
I have cache everything enabled in ewww.io and this will copy the javascript assets ot my CDN. I will need to manually update these js files each time a Matomo update is installed.
I change my Matomo tracker code to include the new CDN location
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | <!-- Matomo --> <script type="text/javascript"> var _paq = _paq || []; /* tracker methods like "setCustomDimension" should be called before "trackPageView" */ _paq.push(['trackPageView']); _paq.push(['enableLinkTracking']); (function() { var u="//fearby.com/utils/matomo/"; _paq.push(['setTrackerUrl', u+'piwik.php']); _paq.push(['setSiteId', '1']); var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; g.type='text/javascript'; g.async=true; g.defer=true; g.src='https://fearby-com.exactdn.com/wp-content/piwik.js'; s.parentNode.insertBefore(g,s); })(); </script> <!-- End Matomo Code --> |
Advertisement:
I could not find out how to change the location of my (now CDN cached https://fearby-com.exactdn.com/wp-content/optOut.js) so I temporarily disabled the opt-out form on my front page.
todo: Find out how to change the CDN location of optOut.js and re-enabled the form.
All assets are loading from CDN.
Analytics Reporting
Graphs are not as pretty as Google Analytics but they are working.
Mobile Reporting
Mobile reporting is good too.
Good luck and I hope this guide helps someone
Ask a question or recommend an article
Revision History
v1.0 Initial post
I recently had an issue where I set up a website for a friend. I invested 6 hours into setting up, the server and backups were automatically deleted after 7 days while I was away from my keyboard because I assumed the account was valid and had credits.
Read Full Article:
No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider
Advertisement:
Aside
If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.
Buy a domain name here
Now on with the post.
Murphy’s Law
I recently had an issue where I set up a website for a friend. I invested 6 hours into setting up..
I setup…
I had tested GTMetrix scores = less than 1 second. Security headers were tested and I was happy with the site.
The server and backups were automatically deleted after 7 days while I was away from my keyboard because I assumed the account was valid and had credits.
Lesson Learned
I have guides on setting up a server on UpCloud, AWS, Vultr, Digital Ocean but setting up can be rather repetitive so how can you prevent resetting up servers?
Why Plan for the Worst
How I will prevent this in future
I hope this guide helps someone.
Please consider using my referral code and get $25 UpCloud VM credit if you need to create a server online.
https://www.upcloud.com/register/?promo=D84793
Ask a question or recommend an article
Revision History
v1.0 Initial Post
This is how I set up a dedicated Debian subdomain (VM), Installed MySQL 14 and connected to it from a WordPress installation on a different VM
Advertisement:
Aside
If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving fearby.com from Vultr to UpCloud.
Buy a domain name here
Now on with the post.
Fearby.com
I will be honest, fearby.com is my play server where I can code, learn about InfoSec and share (It’s also my stroke rehab blog).
There is no faster way to learn than actually doing. The problem is my “doing” usually breaks the live site from time to time (sorry).
I really need to set up a testing environment (DEV-TEST-LIVE or GREEN-BLUE) server(s). GREEN-BLUE has advantages as I can always have a hot spare ready. All I need to do is toggle DNS and I can set the GREEN or BLUE server as the live server.
But first I need to separate my database from my current fearby.com server and setup up a new web server. Having a Green and Blue server that uses one database server will help with near real-time production website switches.
Dedicated Database Server
I read the following ( Should MySQL and Web Server share the same server? ) at Percona Database Performance Blog. Having a separate database server should not negatively impact performance (It may even help improve speeds).
Deploy a Debian VM (not Ubuntu)
I decided to set up a Debian server instead of Ubuntu (mostly because of the good focus on stability and focus on security within Debian).
I logged into the UpCloud dashboard on my mobile phone and deployed a Debian server in 5 mins. I will be using my existing how to setup Ubuntu on UpCloud guide (even though this is Debian).
TIP: Sign up to UpCloud using this link to get $25 free UpCloud VM credit.
Deploy a Debian server setup steps:
After 5 mins your server should be deployed.
Advertisement:
After Deploy
Setup DNS
Login to your DNS provider and create DNS records to the new IP’s (IPv4 and IPv6) provided by UpCloud. It took DNS 12 hours to replicate to my in Australia.
Setup a Firewall (at UpCloud)
I would recommend you set up a firewall at UpCloud as soon as possible (don’t forget to add the recommended UpCloud DNS IP’s and any whitelisted IP’s your firewall).
Block everything and only allow
Read my post on setting up a whitelisted IP on an UpCloud VM… as it is a good idea.
UpCloud thankfully has a copy firewall feature that is very handy.
After I set up the firewall I SSH’ed into my server (I use vSSH on OSX buy you could use PUTTY).
I updated the Debian system with the following command
1 | sudo apt update |
Get the MySQL Package
Visit http://repo.mysql.com/ and get the URL of the latest apt-config repo deb file (e.g “mysql-apt-config_0.8.9-1_all.deb”). Make a temp folder.
1 2 | mkdir /temp cd /temp |
Download the MySQL deb Package
1 | wget http://repo.mysql.com/mysql-apt-config_0.8.9-1_all.deb |
Install the package
1 | sudo dpkg -i mysql-apt-config_0.8.9-1_all.deb |
Update the system again
1 | sudo apt update |
Install MySQL on Debian
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | sudo apt install mysql-server Reading package lists... Done Building dependency tree Reading state information... Done The following additional packages will be installed: libaio1 libatomic1 libmecab2 mysql-client mysql-common mysql-community-client mysql-community-server psmisc The following NEW packages will be installed: libaio1 libatomic1 libmecab2 mysql-client mysql-common mysql-community-client mysql-community-server mysql-server psmisc 0 upgraded, 9 newly installed, 0 to remove and 1 not upgraded. Need to get 37.1 MB of archives. After this operation, 256 MB of additional disk space will be used. Do you want to continue? [Y/n] y Get:1 http://repo.mysql.com/apt/debian stretch/mysql-5.7 amd64 mysql-community-client amd64 5.7.22-1debian9 [8886 kB] Get:2 http://deb.debian.org/debian stretch/main amd64 mysql-common all 5.8+1.0.2 [5608 B] Get:3 http://deb.debian.org/debian stretch/main amd64 libaio1 amd64 0.3.110-3 [9412 B] Get:4 http://deb.debian.org/debian stretch/main amd64 libatomic1 amd64 6.3.0-18+deb9u1 [8966 B] Get:5 http://deb.debian.org/debian stretch/main amd64 psmisc amd64 22.21-2.1+b2 [123 kB] Get:6 http://deb.debian.org/debian stretch/main amd64 libmecab2 amd64 0.996-3.1 [256 kB] Get:7 http://repo.mysql.com/apt/debian stretch/mysql-5.7 amd64 mysql-client amd64 5.7.22-1debian9 [12.4 kB] Get:8 http://repo.mysql.com/apt/debian stretch/mysql-5.7 amd64 mysql-community-server amd64 5.7.22-1debian9 [27.8 MB] Get:9 http://repo.mysql.com/apt/debian stretch/mysql-5.7 amd64 mysql-server amd64 5.7.22-1debian9 [12.4 kB] Fetched 37.1 MB in 12s (3023 kB/s) Preconfiguring packages ... Selecting previously unselected package mysql-common. (Reading database ... 34750 files and directories currently installed.) Preparing to unpack .../0-mysql-common_5.8+1.0.2_all.deb ... Unpacking mysql-common (5.8+1.0.2) ... Selecting previously unselected package libaio1:amd64. Preparing to unpack .../1-libaio1_0.3.110-3_amd64.deb ... Unpacking libaio1:amd64 (0.3.110-3) ... Selecting previously unselected package libatomic1:amd64. Preparing to unpack .../2-libatomic1_6.3.0-18+deb9u1_amd64.deb ... Unpacking libatomic1:amd64 (6.3.0-18+deb9u1) ... Selecting previously unselected package mysql-community-client. Preparing to unpack .../3-mysql-community-client_5.7.22-1debian9_amd64.deb ... Unpacking mysql-community-client (5.7.22-1debian9) ... Selecting previously unselected package mysql-client. Preparing to unpack .../4-mysql-client_5.7.22-1debian9_amd64.deb ... Unpacking mysql-client (5.7.22-1debian9) ... Selecting previously unselected package psmisc. Preparing to unpack .../5-psmisc_22.21-2.1+b2_amd64.deb ... Unpacking psmisc (22.21-2.1+b2) ... Selecting previously unselected package libmecab2:amd64. Preparing to unpack .../6-libmecab2_0.996-3.1_amd64.deb ... Unpacking libmecab2:amd64 (0.996-3.1) ... Selecting previously unselected package mysql-community-server. Preparing to unpack .../7-mysql-community-server_5.7.22-1debian9_amd64.deb ... Unpacking mysql-community-server (5.7.22-1debian9) ... Selecting previously unselected package mysql-server. Preparing to unpack .../8-mysql-server_5.7.22-1debian9_amd64.deb ... Unpacking mysql-server (5.7.22-1debian9) ... Setting up libatomic1:amd64 (6.3.0-18+deb9u1) ... Setting up psmisc (22.21-2.1+b2) ... Setting up mysql-common (5.8+1.0.2) ... update-alternatives: using /etc/mysql/my.cnf.fallback to provide /etc/mysql/my.cnf (my.cnf) in auto mode Setting up libmecab2:amd64 (0.996-3.1) ... Processing triggers for libc-bin (2.24-11+deb9u3) ... Setting up libaio1:amd64 (0.3.110-3) ... Processing triggers for systemd (232-25+deb9u4) ... Processing triggers for man-db (2.7.6.1-2) ... Setting up mysql-community-client (5.7.22-1debian9) ... Setting up mysql-client (5.7.22-1debian9) ... Setting up mysql-community-server (5.7.22-1debian9) ... update-alternatives: using /etc/mysql/mysql.cnf to provide /etc/mysql/my.cnf (my.cnf) in auto mode Created symlink /etc/systemd/system/multi-user.target.wants/mysql.service -> /lib/systemd/system/mysql.service. Setting up mysql-server (5.7.22-1debian9) ... Processing triggers for libc-bin (2.24-11+deb9u3) ... Processing triggers for systemd (232-25+deb9u4) ... |
Secure MySQL
Advertisement:
You can secure the MySQL server deployment (set options as needed)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | sudo mysql_secure_installation Enter password for user root: ******************************************** VALIDATE PASSWORD PLUGIN can be used to test passwords and improve security. It checks the strength of password and allows the users to set only those passwords which are secure enough. Would you like to setup VALIDATE PASSWORD plugin? Press y|Y for Yes, any other key for No: No Using existing password for root. Change the password for root ? ((Press y|Y for Yes, any other key for No) : No ... skipping. By default, a MySQL installation has an anonymous user, allowing anyone to log into MySQL without having to have a user account created for them. This is intended only for testing, and to make the installation go a bit smoother. You should remove them before moving into a production environment. Remove anonymous users? (Press y|Y for Yes, any other key for No) : Yes Success. Normally, root should only be allowed to connect from 'localhost'. This ensures that someone cannot guess at the root password from the network. Disallow root login remotely? (Press y|Y for Yes, any other key for No) : No ... skipping. By default, MySQL comes with a database named 'test' that anyone can access. This is also intended only for testing, and should be removed before moving into a production environment. Remove test database and access to it? (Press y|Y for Yes, any other key for No) : Yes - Dropping test database... Success. - Removing privileges on test database... Success. Reloading the privilege tables will ensure that all changes made so far will take effect immediately. Reload privilege tables now? (Press y|Y for Yes, any other key for No) : Yes Success. All done! |
Install NGINX
I installed NGINX to allow Adminer MySQL GUI to be used
I ran these commands to install NGINX.
1 2 3 | sudo apt update sudo apt upgrade sudo apt-get install nginx |
I edited my NGINX configuration as required.
I reloaded NGINX
1 2 3 | sudo nginx -t sudo nginx -s reload sudo systemctl restart nginx |
Advertisement:
Install PHP
I followed this guide to install PHP on Debian.
1 2 3 4 5 6 7 8 9 10 | sudo apt update sudo apt upgrade sudo apt install ca-certificates apt-transport-https wget -q https://packages.sury.org/php/apt.gpg -O- | sudo apt-key add - echo "deb https://packages.sury.org/php/ stretch main" | sudo tee /etc/apt/sources.list.d/php.list sudo apt update sudo apt install php7.2 sudo apt install php-pear php7.2-curl php7.2-dev php7.2-mbstring php7.2-zip php7.2-mysql php7.2-xml php7.2-cli php7.2-common |
Install PHP FPM
1 | apt-get install php7.2-fpm |
Increase Upload Limits
You may need to temporarily increase upload limits in NGINX and PHP before you can restore a WordPress database. My feabry.com blog is about 87MB.
Add “client_max_body_size 100M;” to “/etc/nginx/nginx.conf”
Add the following to “/etc/php/7.2/fpm/php.ini”
Restore a backup of my MySQL database in MySQL
You can now use Adminer to restore your blog to MySQL. Read my post here on Adminer here. I used Adminer to move my WordPress site from CPanel to a self-managed server a year ago.
First login to your source server and export your desired database then login to the target server and import the database.
Firewall Check
Don’t forget to allow your WordPress site’s 2x Public IP’s and 1x Private IP to access port 3306 in your UpCloud Firewall.
How to check open ports on your current server
1 | sudo netstat -plunt |
Set MySQL Permissions
Open MySQL
1 | mysql --host=localhost --user=root --password=*************************************************************************** |
I ran these statements to grant the user logging in on the nominate IP’s access to MySQL.
1 2 3 4 5 | mysql> GRANT ALL ON databasenmae.* TO databaseusername@IPv4Server1PublicAddress IDENTIFIED BY '***********sql*user*password*************'; GRANT ALL ON databasenmae.* TO databaseusername@IPv4Server1PrivateAddress IDENTIFIED BY '***********sql*user*password*************'; GRANT ALL ON databasenmae.* TO databaseusername@IPv4Server2PublicAddress IDENTIFIED BY '***********sql*user*password*************'; GRANT ALL ON databasenmae.* TO databaseusername@IPv4Server1PrivateAddress IDENTIFIED BY '***********sql*user*password*************'; |
Reload permissions in MySQL
1 | FLUSH PRIVILEGES; |
Allow access to the Debian machine from known IP’s
Edit “/etc/host.allow”
Additions (known safe IP’s that need access to this MySQL remotely).
1 2 3 4 5 6 | mysqld : IPv4Server1PublicAddress : allow mysqld : IPv4Server1PrivateAddress : allow mysqld : IPv4Server2PublicAddress : allow mysqld : IPv4Server1PrivateAddress : allow mysqld : ALL : deny |
Tell MySQL to listen on
Edit “/etc/mysql/my.cnf”
Added..
1 2 3 4 5 6 7 8 9 10 | [mysqld] user = mysql pid-file = /var/run/mysqld/mysqld.pid socket = /var/run/mysqld/mysqld.sock port = 3306 basedir = /usr datadir = /var/lib/mysql tmpdir = /tmp language = /usr/share/mysql/English bind-address = DebianServersIntenalIPv4Address |
I guess you could change the port to something random???
Restart MySQL
1 | sudo service mysql restart |
Install a second local firewall on Debian
Advertisement:
Install ufw
1 | sudo apt-get instal ufw |
Do add the IP of your desired server or VPN to access SSH
1 | sudo ufw allow from 123.123.123.123 to any port 22 |
Do add the IP of your desired server or VPN to access WWW
1 | sudo ufw allow from 123.123.123.123 to any port 80 |
Now add the known IP’s (e.g any web servers public (IPv4/IPv6) or Private IP’s) that you wish to grant access to MySQL (e.g the source website that used to have MySQL)
1 | sudo ufw allow from 123.123.123.123 to any port 3306 |
Do add UpCloud DNS Servers to your firewall
1 2 3 4 | sudo ufw allow from 94.237.127.9 to any port 53 sudo ufw allow from 94.237.40.9 to any port 53 sudo ufw allow from 2a04:3544:53::1 to any port 53 sudo ufw allow from 2a04:3540:53::1 to any port 53 |
Add all other rules as needed (if you stuff up and lock your self out you can login to the server with the Console on UpCloud)
Restart the ufw firewall
1 2 | sudo ufw disable sudo ufw enable |
Prevent MySQL starting on the source server
Now we can shut down MySQL on the source server (leave it there just in case).
Edit “/etc/init/mysql.conf”
Comment out the line that contains “start on ” and save the file
and run
1 | sudo systemctl disable mysql |
Reboot
1 | shutdown -r now |
Stop and Disable NGINX on the new DB server
We don’t need NGINX running now the database has been imported with Adminer.
Stop and prevent NGINX from starting up on startup.
1 2 3 | /etc/init.d/nginx stop sudo update-rc.d -f nginx disable sudo systemctl disable nginx |
Check to see if MySQL is Disabled
1 2 3 4 | service mysql status * mysql.service - MySQL Community Server Loaded: loaded (/lib/systemd/system/mysql.service; disabled; vendor preset: enabled) Active: inactive (dead) |
Yep
Test access to the database server in PHP code
Add to dbtest.php
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | <em>SELECT guid FROM wp_posts</em>()<br /> <ul><?php //External IP (charged after quota hit) //$servername = 'db.yourserver.com'; //Private IP (free) //$servername = '10.x.x.x'; $username = 'username'; $password = '********your*password*********'; $dbname = 'database'; // Create connection $conn = new mysqli($servername, $username, $password, $dbname); // Check connection if ($conn->connect_error) { die("Connection failed: " . $conn->connect_error); } $sql = 'SELECT guid FROM wp_posts'; $result = $conn->query($sql); if ($result->num_rows > 0) { // output data of each row while($row = $result->fetch_assoc()) { echo $row["guid"] . "<br>"; } } else { echo "0 results"; } $conn->close(); ?></ul> Done |
Check for open ports.
You can install nmap on another server and scan for open ports
Advertisement:
Install nmap
1 | sudo apt-get install nmap |
Scan a server for open ports with nmap
You should see this on a server that has access to see port 3306 (port 3306 should not be visible by non-whitelisted IP’s). Port 3shouldoudl not be seen via everyone.
1 2 3 4 5 6 7 8 9 | sudo nmap -PN db.yourserver.com Starting Nmap 7.40 ( https://nmap.org ) at 2018-07-20 14:15 UTC Nmap scan report for db.yourserver.com (IPv4IP) Host is up (0.0000070s latency). Other addresses for db.yourserver.com (not scanned): IPv6IP Not shown: 997 closed ports PORT STATE SERVICE 3306/tcp open mysql |
You should see something like this on a server that has access to see port 80/443 (a web server)
1 2 3 4 5 6 7 8 9 10 | sudo nmap -PN yourserver.com Starting Nmap 7.40 ( https://nmap.org ) at 2018-07-20 14:18 UTC Nmap scan report for db.yourserver.com (IPv4IP) Host is up (0.0000070s latency). Other addresses for db.yourserver.com (not scanned): IPv6IP Not shown: 997 closed ports PORT STATE SERVICE 80/tcp open http 443/tcp open https |
I’d recommend you use a service like https://pentest-tools.com/network-vulnerability-scanning/tcp-port-scanner-online-nmap# to check for open ports. https://hackertarget.com/tcp-port-scan/ is a great tool too.
https://www.infobyip.com/tcpportchecker.php is also a free port checker that you can use to verify individual closed ports.
Hardening MySQL and Debian
Read: https://www.debian.org/doc/manuals/securing-debian-howto/ap-checklist.en.html
Configuring WordPress use the dedicated Debian VM
On the source server that used to have MySQL edit your wp-config.php file for WordPress.
Remove
1 | define('DB_HOST', 'localhost'); |
add (read the update below, I changed the DNS IP to the Private IP to have free traffic)
1 2 3 4 5 6 7 8 | //Oriinal localhost //define('DB_HOST', 'localhost'); //New external host via DNS (Charged after quota hit) //define('DB_HOST', 'db.fearby.com'); //New external host via Private IP (Free) define('DB_HOST','10.x.x.x'); |
Restart NGINX
1 2 3 | sudo nginx -t sudo nginx -s reload sudo systemctl restart nginx |
Restart PHP-FPM
1 | service php7.2-fpm restart |
Conclusion
Nice, I seem to have shaved off 0.3 seconds in load times (25% improvement)
Update: Using a Private IP or Public IP between WordPress and MySQL servers
After I released this blog post (version 1.0 with no help from UpCloud) UpCloud contacted me and said the following.
1 2 3 4 5 6 7 8 9 10 | Hello Simon, I notice there's no mention of using the private network IPs. Did you know that we automagically assign you one when you deploy with our templates. The private network works out of the box without additional configuration, you can use that communicate between your own cloud servers and even across datacentres. There's no bandwidth charge when communicating over private network, they do not go through public internet as well. With this, you can easily build high redundant setups. Let me know if you have any other questions. -- Kelvin from UpCloud |
I will have updated my references in this post and replace the public IP address (that is linked to DNS record for db.fearby.com) and instead use the private ip address (e.g 10.x.x.x), your servers private IP address is listed against the public IPv$ and IPv6 address.
I checked that the local ufw firewall did indeed allow the private IP access to MySQL.
1 2 | sudo ufw status numbered |grep 10.x.x.x [27] 3306 ALLOW IN 10.x.x.x |
On my new Debian MySQL server, I edited the file /etc/mysql/my.cnf and changed the IP to the private IP and not the public IP.
Now it looked like
1 2 3 4 5 6 7 8 9 10 | [mysqld] user = mysql pid-file = /var/run/mysqld/mysqld.pid socket = /var/run/mysqld/mysqld.sock port = 3306 basedir = /usr datadir = /var/lib/mysql tmpdir = /tmp language = /usr/share/mysql/English bind-address = 10.x.x.x |
(10.x.x.x my Debian servers private IP)
On my WordPress instance, I edited the file /www-root/wp-config.php
I added the new private host
1 2 3 4 5 6 7 8 | //Oriinal localhost //define('DB_HOST', 'localhost'); //New external host via DNS (Charged after quota hit) //define('DB_HOST', 'db.fearby.com'); //New external host via Private IP (Free) define('DB_HOST','10.x.x.x'); |
(10.x.x.x my Debian servers private IP)
Alos on Debian/MySQL ensure you have granted access to the private IP of the WordPress server
Edit /etc/host.allow
Add
1 | mysqld : 10.x.x.x : allow |
Restart MySQL
1 | sudo systemctl restart mysql |
TIP: Enable UpCloud Backups
Do setup automatic backups (and or take manual backups). Backups are an extra charge but are essential IMHO.
Troubleshooting
If you can’t access MySQL log back into MySQL
1 | mysql --host=localhost --user=root --password=*************************************************************************** |
and run
1 | GRANT ALL PRIVILEGES ON *.* TO databaseuser@'%' IDENTIFIED BY '***********sql*user*password*************''; FLUSH PRIVILEGES; |
Reboot
Lower Upload Limits
Don’t forget to lower file upload sizes in NGINX and PHP (e.g 2M) now that the database has been restored.
I hope this guide helps someone.
TIP: Sign up to UpCloud using this link to get $25 free UpCloud VM credit.
https://www.upcloud.com/register/?promo=D84793
Ask a question or recommend an article
Advertisement:
Revision History
v1.6 Changed Public IP use to private IP use to ensure we are not charged when the serves sage goes over the quota
v1.5 Fixed 03 type (should have been 0.3)
v1.4 added disable nginx info
v1.3 added https://www.infobyip.com/tcpportchecker.php
v1.1 added https://hackertarget.com/tcp-port-scan/
v1.0 Initial Post
This post will explain why I will never buy a new Apple Laptop until they fix the hardware cooling issues.
This post will explain why I will never buy a new Apple Laptop until they fix the hardware cooling issues.
Background
I used to work in retail selling computers and I would go to great lengths to open a desktop computer chassis and talk someone out of buying a cheaper/slower computer (usually when it had a Cyrix Media GX processor in it). I would do myself out of higher commission and burn time educating customers. I have blogged about what to look for when buying a computer (here).
2012
In 2012 I bought my first Apple Mac computer to write iOS apps (write your first OSX app). I would call myself an Apple fanboy (previously being a PC fanboy for 15 years). I have never rebuilt my OSX system in 6 years buy would rebuild Windows every 6 months. Some Apple things I like.
2017
My Mid 2012 Mac Book Pro i7 processor overheats like crazy. I have blogged about my Mid 2012 MPB overheating issues (read here). I have even gone and installed third party software to control the speeds of my Mac’s fans (read here).
Inside my Mid 2012 Mac Book Pro (heatsink and fans at the top)
Stupidly thin heatsink (IMHO).
Complete heatsink (CPU and GPU plate)
I am certain this Mac Book heatsink is too small for the processor and graphics card.
As I type this my Mac Book Pro is Thermal throttling (slowing down the CPU) while typing a blog post (not gaming).
My only option is to crank up the fans to 100% and overrise Apple silence first mantra.
I am currently sitting here at Winter with my MBP 2012 MBP i7 fans running at 100% to try (try) and prevent thermal throtelling killing my productivity. https://t.co/IM6IlnmjC7
— Simon Fearby (Aussie DevSecOps) (@FearbySoftware) July 18, 2018
Intel Power Gadget showing thermal throttling (CPU dropping t0 almost 1Ghz to drop temps).
Move forward to 2018
Today I learned that Apple is putting an Intel i9 Procesor into a laptop, great? Hold onto your cash, that thing will run very hot and will never operate at its maximum potential.
Reviews are scathing.
I tweeted..
What a joke, why is @Apple putting an Intel i9 into a stupidly thin Mac Book Pro, my i7 can barely keep cool https://t.co/IM6IlnmjC7
— Simon Fearby (Aussie DevSecOps) (@FearbySoftware) July 13, 2018
Apple’s Website: https://www.apple.com/macbook-pro/
What a waste of a good processor.
Below you will see the fallout on YouTube from Apple putting an i9 Processor in the latest 15″ Mac Book Pros.
Dave Lee posted “MacBook Pro 15 (2018) – Beware the Core i9”
TechLinked posted “2018 Macbook ALREADY Overheating?”
AppleInsider – 2018 MacBook Pro i9 Thermal Throttling CONFIRMED!
Best of all, Louis Rossmann summed up the Apple situation perfectly.
Update 25th July
Apple is doubling down on the lack of cooling (calling it a “missing digital key”).
I will #BoycottAppleProMachines
That’s all.
Revision History
v1.4 Added update 25th July 2018 Missing Digital Key
v1.3 Gizmodo link
v1.2 Test new db server
v1.1 Added Apple Insider video
v1.0 Initial Post
Here is how I added two subdomains (one pointing to a new UpCloud VM and the other pointing to an NGINX subsite) on Ubuntu 18.04
Advertisement:
If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.
UpCloud performance is great.
Buy a domain name from Namecheap here.
Goal(s)
Setup 2x subdomains on https://fearby.com
– Sub Domain #1: https://test.fearby.com (pointing to a dedicated UpCloud VM in Singapore for testing).
– Sub Domain #2: https://audit.fearby.com (pointing to a sub-website on the NGINX/VM that runs https://fearby.com )
Let’s set up the first Sub Domain (dedicated VM) and SSL
Backup
Do backup your server first.
VM
I created a second server ($5 month or $0.07c hour 1,024MB Memory, 25GB Disk, 1024 GB Month Data Transfer) at UpCloud. If you don’t already have an account at UpCloud use this link to signup and get $25 free credit ( https://www.upcloud.com/register/?promo=D84793 ). Read my blog post on why UpCloud is awesome and how I moved my domain to UpCloud.
Once I spun up a second server I obtained the IPv4 and IPv6 IP addresses of the new “test” VM from the UpCloud dashboard.
1 2 | IPV4 IP: 94.237.65.54 IPV6 IP: 2a04:3543:1000:2310:24b7:7cff:fe92:468c |
DNS
These DNS records were already in place with my DNS provider (Cloudflare).
1 2 | A fearby.com 209.50.48.88 AAAA fearby.com 2605:7380:1000:1310:24b7:7cff:fe92:0d64 |
I added these DNS records for the subdomains.
I added a new A NAME record for the new shared NGINX subdomain (for https://audit.fearby.com), this subdomain will be a sub-website that is running off the same server as https://fearby.com
1 2 | A audit 209.50.48.88 AAAA audit 2605:7380:1000:1310:24b7:7cff:fe92:0d64 |
I added another set of records for the new dedicated VM subdomain (for https://test.fearby.com)
1 2 | A test 94.237.65.54 AAAA test 2a04:3543:1000:2310:24b7:7cff:fe92:468c |
I waited for DNS to replicate around the globe by watching https://www.whatsmydns.net/
Setup a Firewall
On the new dedicated https://test.fearby.com VM, I installed the ufw firewall.
1 | sudo apt-get install ufw |
I configured the firewall to allow minimum ports (and added whitelisted IP for port 22 and added UpCloud DNS servers). I will lock this down some more later.
TIP: If your ISP does not offer a dedicated IP try a VPN. I use https://cyberghostvpn.com on OSX and Android.
Firewall rules.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | sudo ufw status numbered To Action From -- ------ ---- [ 1] 22 ALLOW IN x.x.x.x [ 2] 80 ALLOW IN Anywhere [ 3] 443 ALLOW IN Anywhere [ 4] 53 ALLOW IN 93.237.127.9 [ 5] 53 ALLOW IN 93.237.40.9 [ 6] 25 DENY IN Anywhere [ 7] 80 (v6) ALLOW IN Anywhere (v6) [ 8] 443 (v6) ALLOW IN Anywhere (v6) [ 9] 53 ALLOW IN 2a04:3540:53::1 [10] 53 ALLOW IN 2a04:3544:53::1 [11] 22 ALLOW IN x.x.x.x.x.x.x.x.x [12] 25 (v6) DENY IN Anywhere (v6) |
I enabled the firewall.
1 | sudo ufw enable |
Install NGINX (on https://test.fearby.com)
On the new dedicated https://test.fearby.com VM I…
Created a new www root
1 | mkdir /www-root |
Set permissions
1 | sudo chown -R www-data:www-data /www-root |
Installed NGINX
1 2 | sudo apt-get update sudo apt-get install nginx |
I created a placeholder webpage
1 | sudo nano /www-root/index.html |
Configured the root value in /etc/nginx/sites-available/default
Created a symbolic link of the nginx config
1 | sudo ln -s /etc/nginx/sites-available/default /etc/nginx/sites-enabled/default |
Lets Encrypt SSL
I have previously setup Lets encrypt on Ubuntu 16.04 but not 18.04. Certbot had info on setting up Lets Encrypt for 14.x 16.x and 17.x but not 18.x
Full credit for the SSL steps goes to @Linuxize ( tips on setting up Lets Encrypt on Ubuntu 18.04 ). Check out https://linuxize.com/
I installed Lets Encrypt certbot
1 2 | sudo apt update sudo apt install certbot |
I created a new Diffie–Hellman key
1 2 | mkdir -p /etc/ssl/certs/ sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048 |
Map requests to http://test.fearby.com/.well-known/acme-challenge to /var/lib/letsencrypt/.well-known ( Read the linuxize post for detailed steps ).
1 2 3 | mkdir -p /var/lib/letsencrypt/.well-known chgrp www-data /var/lib/letsencrypt chmod g+s /var/lib/letsencrypt |
Create a /etc/nginx/snippets/letsencrypt.conf on http://test.fearby.com and enforce the redirect.
1 2 3 4 5 6 | location ^~ /.well-known/acme-challenge/ { allow all; root /var/lib/letsencrypt/; default_type "text/plain"; try_files $uri =404; } |
Create a /etc/nginx/snippets/ssl.conf file on http://test.fearby.com
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | ssl_dhparam /etc/ssl/certs/dhparam.pem; ssl_session_timeout 1d; ssl_session_cache shared:SSL:50m; ssl_session_tickets off; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS'; ssl_prefer_server_ciphers on; ssl_stapling on; ssl_stapling_verify on; resolver 8.8.8.8 8.8.4.4 valid=300s; resolver_timeout 30s; add_header Strict-Transport-Security "max-age=15768000; includeSubdomains; preload"; add_header X-Frame-Options SAMEORIGIN; add_header X-Content-Type-Options nosniff; |
Let’s get a certificate
1 | sudo certbot certonly --agree-tos --email your@email.com --webroot -w /var/lib/letsencrypt/ -d test.fearby.com |
Certificates have been created 🙂
1 2 3 4 5 6 7 8 9 | ls -al /etc/letsencrypt/live/test.fearby.com/ total 12 drwxr-xr-x 2 user user 4096 Jun 26 11:30 . drwx------ 3 user user 4096 Jun 26 11:30 .. -rw-r--r-- 1 user user 543 Jun 26 11:30 README lrwxrwxrwx 1 user user 39 Jun 26 11:30 cert.pem -> ../../archive/test.fearby.com/cert1.pem lrwxrwxrwx 1 user user 40 Jun 26 11:30 chain.pem -> ../../archive/test.fearby.com/chain1.pem lrwxrwxrwx 1 user user 44 Jun 26 11:30 fullchain.pem -> ../../archive/test.fearby.com/fullchain1.pem lrwxrwxrwx 1 user user 42 Jun 26 11:30 privkey.pem -> ../../archive/test.fearby.com/privkey1.pem |
Now lets edit “/etc/nginx/sites-available/default” on https://test.fearby.com VM and add the cert paths.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | server { listen 80 default_server; listen [::]:80 default_server; listen 443 ssl http2; listen [::]:443 ssl http2; if ($scheme != "https") { return 301 https://$host$request_uri; } ssl_certificate /etc/letsencrypt/live/test.fearby.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/test.fearby.com/privkey.pem; ssl_trusted_certificate /etc/letsencrypt/live/test.fearby.com/chain.pem; include snippets/ssl.conf; #ssl_stapling on; # Requires nginx >= 1.3.7 # add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"; add_header X-Frame-Options DENY; add_header X-Content-Type-Options nosniff; add_header X-XSS-Protection "1; mode=block"; root /www-root/; include snippets/letsencrypt.conf; index index.html; server_name test.fearby.com; location / { try_files $uri $uri/ =404; } } |
Advertisement:
Reload NGINX
1 | sudo systemctl reload nginx |
or
1 2 3 | sudo nginx -t sudo nginx -s reload sudo systemctl reload nginx |
Now let’s setup the second subdomain (subsite off https://fearby.com) and SSL
VM
I already have NGINX on https://fearby.com set up a second site.
DNS
We have already setup a DNS record for https://audit.fearby.com (above)
Firewall
Already configured at https://fearby.com
SSL
Because I had an existing Comodo certificate on https://fearby.com I am going to repeat the steps above to generate a new certificate but save the NGINX config to /etc/nginx/sites-available/audit.fearby.com (this activates the second site)
TIP: Follow the Linuxize guide here (for creating ssl.conf, letsencrypt.conf etc config files), Do a backup and restore if need be.
I created a new Diffie–Hellman key
1 2 | mkdir -p /etc/ssl/certs/ sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048 |
Let’s get a certificate
1 | sudo certbot certonly --agree-tos --email your@email.com --webroot -w /var/lib/letsencrypt/ -d audit.fearby.com |
Configure NGINX
Map requests to http://audit.fearby.com/.well-known/acme-challenge to /var/lib/letsencrypt/.well-known ( Read the linuxize post for detailed steps ).
1 2 3 | mkdir -p /var/lib/letsencrypt/.well-known chgrp www-data /var/lib/letsencrypt chmod g+s /var/lib/letsencrypt |
I created a new NGINX site ( /etc/nginx/sites-available/audit.fearby.com )
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | #proxy_cache_path /tmp/nginx-cache keys_zone=one:10m;# server { root /www-audit-root; # Listen Ports listen 80; listen [::]:80; listen 443 ssl http2; listen [::]:443 ssl http2; # Default File index index.html index.php index.htm; # Server Name server_name audit.fearby.com; include snippets/letsencrypt.conf; location / { try_files $uri $uri/ =404; } ssl_certificate /etc/letsencrypt/live/audit.fearby.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/audit.fearby.com/privkey.pem; ssl_trusted_certificate /etc/letsencrypt/live/audit.fearby.com/chain.pem; ssl_dhparam /etc/ssl/certs/auditdhparam.pem; ssl_session_timeout 1d; #ssl_session_cache shared:SSL:50m; ssl_session_tickets off; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA38$ ssl_prefer_server_ciphers on; ssl_stapling on; ssl_stapling_verify on; #resolver 8.8.8.8 8.8.4.4 valid=300s; #resolver_timeout 30s; add_header Strict-Transport-Security "max-age=15768000; includeSubdomains; preload"; add_header X-Frame-Options SAMEORIGIN; add_header X-Content-Type-Options nosniff; if ($scheme != "https") { return 301 https://$host$request_uri; } } |
I created a symbolic link of the config file
1 | sudo ln -s /etc/nginx/sites-available/audit.fearby.com /etc/nginx/sites-enabled/audit.fearby.com |
Reload NGINX
1 | sudo systemctl reload nginx |
or
1 2 3 | sudo nginx -t sudo nginx -s reload sudo systemctl reload nginx |
How to test the certificate renewal
1 | sudo certbot renew --dry-run |
Automate the renewal in crontab (every 12 hours)
I set this crontab entry up on https://fearby.com and https://test.fearby.com
1 2 | crontab -e 0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(3600))' && certbot -q renew --renew-hook "systemctl reload nginx" |
Conclusion
Yes, I haVe 2 subdomains (1x dedicated VM and the other is a sub-website off an existing server) with SSL certificates.
Ping Results
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | ping -c 4 fearby.com PING fearby.com (209.50.48.88): 56 data bytes 64 bytes from 209.50.48.88: icmp_seq=0 ttl=44 time=220.000 ms 64 bytes from 209.50.48.88: icmp_seq=1 ttl=44 time=290.602 ms 64 bytes from 209.50.48.88: icmp_seq=2 ttl=44 time=311.938 ms 64 bytes from 209.50.48.88: icmp_seq=3 ttl=44 time=330.841 ms --- fearby.com ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 220.000/288.345/330.841/41.948 ms ping -c 4 test.fearby.com PING test.fearby.com (94.237.65.54): 56 data bytes 64 bytes from 94.237.65.54: icmp_seq=0 ttl=44 time=333.590 ms 64 bytes from 94.237.65.54: icmp_seq=1 ttl=44 time=252.433 ms 64 bytes from 94.237.65.54: icmp_seq=2 ttl=44 time=271.153 ms 64 bytes from 94.237.65.54: icmp_seq=3 ttl=44 time=292.685 ms --- test.fearby.com ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 252.433/287.465/333.590/30.200 ms ping -c 4 audit.fearby.com PING audit.fearby.com (209.50.48.88): 56 data bytes 64 bytes from 209.50.48.88: icmp_seq=0 ttl=44 time=281.662 ms 64 bytes from 209.50.48.88: icmp_seq=1 ttl=44 time=307.676 ms 64 bytes from 209.50.48.88: icmp_seq=2 ttl=44 time=227.985 ms 64 bytes from 209.50.48.88: icmp_seq=3 ttl=44 time=215.566 ms --- audit.fearby.com ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 215.566/258.222/307.676/37.845 ms |
Webpage Results
Troubleshooting
If you are having troubles generating the initial certificate check that you have not blocked port 80 and don’t have “Strict-Transport-Security” heavers enabled.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | sudo certbot certonly --agree-tos --email your@email.com --webroot -w /var/lib/letsencrypt/ -d g Saving debug log to /var/log/letsencrypt/letsencrypt.log Obtaining a new certificate Performing the following challenges: http-01 challenge for yoursubdomain.domain.com Using the webroot path /var/lib/letsencrypt for all unmatched domains. Waiting for verification... Cleaning up challenges Failed authorization procedure. yoursubdomain.domain.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficzLlmg_w6Tc: q%!(EXTRA string=<html> <head><title>404 Not Found</title></head> <body bgcolor="white"> <center><h1>404 Not Found</h1></center> <hr><center>) IMPORTANT NOTES: - The following errors were reported by the server: Domain: yoursubdomain.domain.com Type: unauthorized Detail: Invalid response from http://yoursubdomain.domain.com/.well-known/acme-challenge/_QA3jblEydx5mE8I8OdRsd2EdHIj4R-przLlmg_w6Tc: q%!(EXTRA string=<html> <head><title>404 Not Found</title></head> <body bgcolor="white"> <center><h1>404 Not Found</h1></center> <hr><center>) To fix these errors, please make sure that your domain name was entered correctly and the DNS A record(s) for that domain contain(s) the right IP address. |
I re-ran the certbot command but pointed to the real /www-root (not/var/lib/letsencrypt/)
Create a new
1 2 3 | mkdir /www-root/.well-known/ mkdir /www-root/.well-known/acme-challenge/ sudo certbot certonly --agree-tos --email your@email.com --webroot -w /www-root -d yoursubdomain.domain.com |
Advertisement:
I hope this guide helps someone.
Please consider using my referral code and get $25 credit for free.
https://www.upcloud.com/register/?promo=D84793
Ask a question or recommend an article
Revision History
v1.1 Troubleshooting
v1.0 Initial Post
This is how I upgraded from my standalone 1Password 6.x family licence to a 1Password 7 cloud subscription on OSX. I am not reviewing 1Password here.
Advertisement:
This is NOT a paid endorsement, this is output from my legitimate quest from upgrading an old stand-alone family licence to a cloud subscription. I have been using 1Password for the past 5 years and have recommended it to everyone I know.
Always backup your data before updating (things can go wrong), good luck. At the time of writing 1Password 7 was not out of beta.
Why
I have a number of guides on moving away from CPanel, Setting up VM’s on UpCloud, AWS, Vultr or Digital Ocean and let’s say 1Password has helped me store everything from service invoices, SSH password service passwords etc. I did have a stroke last year (caused by the flu (a cough) and luckily all is OK ) and I now realise that having everything out of my brain and in a secure vault is a good idea (touch wood).
Reasons why I use 1Password – Password Manager.
1Password 7 News
1Password 7 has been hitting my twitter timeline, should I upgrade? Here is the official upgrade guide.
Security Researcher Troy Hunt’s – https://haveibeenpwned.com/ is now a feature in 1Password 7
@1Password just keeps getting better and better. Ping: @troyhunt pic.twitter.com/qTtE6XyoXb
— Grant Harrington (@harringg) May 22, 2018
I wrote a PHP implementation to check a password exposure level with Troy Hunt’s pwned passwords API and know it’s a good idea.
Also, there are loads of great features in 1Password 7.
Anything that can help create secure passwords is a good idea.
86% of Passwords are Terrible (and Other Statistics) https://t.co/pSqbb7IV0g by @troyhunt
— Particular Software (@ParticularSW) May 25, 2018
1Password Twitter Support Shoutout
Before I begin I would like to acknowledge the patient 1Password support team on twitter. They answered well over 20 questions from me and handled my frustrations of there not being a clear standalone family licence, I suspected a plot to force people onto a cloud subscription at first.
In an ideal world upgrading, 1Password should be an easy process (1Password Twitter Support indicated)
Load’s of 1 Password activity on Twitter
Phew! ? 1Password 7 for Mac has generated a lot of excitement! If you have any questions, be sure to check out our forums to see if they’ve been answered. Our team is always here. https://t.co/Xixe8e80yY
— 1Password (@1Password) May 23, 2018
Before I downloaded the latest 1Password 7 I fired heaps of questions at the twitter support. I hope 1Password give them a raise or bonus.
I did spend way too long reading past the negative 1Password support posts on “where is the standalone licence”, “beta discounts are gone”, “why so expensive” and “how can I upgrade from 1password 6 and still use dropbox” etc.
I ended up logging a support ticket (looking for the unicorn beta tester discount/stand-alone licence, I think I was too late to join the beta program).
I backed up my 1Password 6 data
Always take backups of your data before upgrading anything.
I also backed up the 1Password file in Dropbox before upgrading. Simply drag it to your desktop.
Advertisement:
I visited https://1password.com/extlink/signin/ and…
Then I..
I did have a peek at the 1Password SSL certificate strength and other tools and they came up all good (I don’t want to use an insecure service).
You too can test SSL on sites with https://dev.ssllabs.com/ssltest/
The only concern I have is TLS 1.3 is not an option yet. I use it on my blog’s web server (guide here) also a few SSL labs identified weak cyphers are presented as available from the server (Is this an issue)?
I also had a look at Google Chrome’s developer console to see if anything out of the ordinary was popping up? The console appears a little chatty? TLS 1.2 was in force securing the client/server communications so that’s nice.
Now that I am logged into my cloud 1Password (trial) account I can…
Now I can connect my new 1Password cloud account to my local 1Password 6 installation by.
Advertisement:
Now I am prompted to import my local 1Password data into the cloud account from my local 1Password.
When the import completed I was prompted to delete the local vault (I said yes because I backed it up).
Tip: 1Password 6 on my Mac did not appear to delete the Dropbox data so I deleted this manually.
After a few minutes, I noticed Dropbox was still syncing files?
Troubleshooting: I had to set my new cloud vault as the primary vault to save to and not the old vault that was syncing via Dropbox. I also deleted all links to Dropbox on iOS and Android devices.
I did notice that 1Password was 6.8.9 (I thought 1password 7 was the latest?, I did try the update button). I ended up ticking “Include beta builds” and then 1Password 7.0 is a download option (maybe this will change in the next few days)?
I opened 1Password 7 on my local desktop.
I had a quick look around in 1Password 7 for the https://haveibeenpwned.com/ feature. I opened an existing account I added to 1Password. It look’s nice.
Some nice alerts and features I noticed when viewing my data in 1 Password 7.
Aside: I had to opt into beta builds on Windows to get 1Password 7 too.
Summary
When I set out and wanted a stand-alone licence but it appears I would need to pay for a licence on Windows and Mac and portable devices.
I overlooked an earlier DM from 1Password (that provided the purchase links) so I decided to go with a subscription (I think I missed the BETA program too, no reply from the hockey app email when opting into beta on Windows).
Buy standalone licences
From what I could see standalone licences only work via Dropbox (or locally) and not via the 1Password cloud.
However, the subscription does away with the requirement to buy multiple licences (all apps are free once you subscribe). I am not sure when 1Password 8 is coming out so I think it is wiser to go with a yearly subscription (that’s about 10.8c a day in Australian peso’s).
Time to Subscribe
I pulled the trigger and subscribed 🙂
Advertisement:
One nice thing is the trial time is added on to the subscription length so if you have 30 days left in the trial it add’s on to the yearly subscription length (13 months), that’s nice.
Conclusion
Happy = Yes (they are shooting fish in a barrel)
Could have been easier to upgrade from 1 password = Yes
I hope this guide helps someone.
Find out more about 1Password at http://1password.com/
Ask a question or recommend an article
Revision History
v1.3 Fixed typo in the title/url.
v1.2 Added Links
v1.1 Added Conclusion
v1.0 Initial post
This guide will show you how you can open a Windows 10 Boot Camp Installation on OSX in Parallels (like a VM).
Advertisement:
Installing Parallels on a Mac allows you to install Windows in A VM, this is handy but you may want to install Windows on a Mac drive with Boot camp (guide here) for better performance.
Can you load this VM-less Windows install in OSX rather than reboot it, the answer is YES (with Parallels v13).
Setup your Windows Bootcamps (see my guide here).
Create a new VM image in Parallels (Select Boot Camp)
Click Continue
Confirm the reaction warning.
Name the VM and choose a location
Set desired memory etc.
Choose your desired clipboard and disk access settings.
Done, now Parallels will prepare your VM (Really Boot Camp)
Preparing
Parallel tools will be automatically installed.
Done, you will now be able to load your Apple Bootcamp partition as it is was a VM inside OSX (or boot it)
yes, the VM file is pointing to the Boot Camp partition.
I hope this guide helps someone.
Ask a question or recommend an article
Revision History
v1.0 Initial post
