Code, Security and Server Stuff

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View All Posts.

on

Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

by

This guide will show how you can set up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

Advertisement:



I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. This post will show how to let Cloudflare handle the DNS for the domain.

Update 2018: For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

Snip from here “Cloudflare’s enterprise-class web application firewall (WAF) protects your Internet property from common vulnerabilities like SQL injection attacks, cross-site scripting, and cross-site forgery requests with no changes to your existing infrastructure.”

Buy a Domain 

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Cloudflare Benefits (Free Plan)

  • DDoS Attack Protection (Huge network to absorb attacks DDoS attacks over 600Gbps are no problem for our 15 Tbps networks)
  • Global CDN
  • Shared SSL certificate (I disabled this and opted to use my own)
  • Access to audit logs
  • 3 page rules (maximum)

View paid plan options here.

Cloudflare CDN map

Cloudflare CDN says it can load assets up to 2x faster, 60% less bandwidth from your servers by delivering assets from 127 data centres.

Cloudflare Global Network

Setup

You will need to sign up at cloudflare.com

Cloudflare

After you create an account you will be prompted to add a siteAdd SiteCloudflare will pull your public DNS records to import.

Query DNS

You will be prompted to select a plan (I selected free)

Plan Select

Verify DNS settings to import.

DNS Import

You will now be asked to change your DNS nameservers with your domain reseller

DNS Nameservers

TIP: If you have an SSL cert (e.g Lets Encrypt) already setup head to the crypto section and select ” Full (Strict)” to prevent ERR_TOO_MANY_REDIRECTS errors.

Strict SSL

Cloudflare UI

I asked Twitter if they could kindly load my site so I could see if Cloudflare dashboard/stats were loading.

The Cloudflare CTO responded.  🙂

Confirm Cloudflare link to a domain from the OSX Comand line

Caching Rule

I set up the following caching rule to cache everything for 8 hours instead of WordPress pages

Page Rules

“fearby.com.com/wp-*” Cache level: Bypass

“fearby.com.com/wp-admin/post.php*” Cache level: Bypass

“fearby.com/*” Cache Everything, Edge Cache TTL: 8 Hours

Cache Results

Cache appears to be sitting at 50% after 12 hours.  having cache os dynamic pages out there is ok unless I need to fix a typo, then I need to login to Cloudflare and clear the cache manually (or wait 8 hours)

Performance after a few hours

DNS times in gtmetrix have now fallen to a sub 200ms (Y Slow is now a respectable A, it was a C before).  I just need to wait for caching and minification to kick in.

DNS Improved

webpagetest.org results are awesome

See here: https://www.webpagetest.org/result/180314_PB_7660dfbe65d56b94a60d7a604ca250b3/

  • Load Time: 1.80s
  • First Byte 0.176s
  • Start Render 1.200s

webpagetest

Google Page Speed Insights Report

Mobile: 78/100

Desktop: 87/100

Check with https://developers.google.com/speed/pagespeed/insights/

Update 24th March 2018 Attacked?

I noticed a spike in and traffic (incoming and threats) on the 24th of March 2018.

I logged into Cloudflare on my mobile device and turned on Under Attack Mode.

Under Attack Flow

Cloudflare was now adding a delay screen in the middle of my initial page load. Read more here.  A few hours after the Attach started it was over.

After the Attack

I looked at the bandwidth and found no increase in traffic from my initial host VM. Nice.

cloudflare-attack-001

Thanks, Cloudflare.

Cloudflare Pros

  • Enabling Attack mode was simple.
  • Soaked up an attack.
  • Free Tier
  • Many Reports
  • Option to force HTTPS over HTTP
  • Option to ban/challenge suspicious IP’s and set challenge timeframes.
  • Ability to setup IP firewall rules and Application Firewalls.
  • User-agent blocking
  • Lockdown URL’s to IP’s (pro feature)
  • Option to minify Javascript, CSS and HTML
  • Option to accelerate mobile links
  • Brotli compression on assets served.
  • Optio to enable BETA Rocket loader for Javascript performance tweaks.
  • Run Javascript service workers from the 120+ CDN’s
  • Page/URL rules o perform custom actions (redirects, skip cache, Encryption etc)
  • HTTP/2 on, IPV6 ON
  • Option to setup load balancing/failover
  • CTO of Cloudflare responded in Twitter 🙂
  • Option to enable rate limiting (charged at 10,000 hits for $0.05c)
  • Option to block countries (pro feature)
  • Option to install apps in Cloudflare like(Goole Analytics,

Cloudflare Cons

  • No more logging into NameCheap to perform DNS management (I now goto Cloudflare, Namecheap are awesome).
  • Cloudflare Support was slow/confusing (I ended up figuring out the redirect problem myself).
  • Some sort of verify Cloudflare Setup/DNS/CDN access would be nice. After I set this up my gtmetrix load times were the same and I was not sure if DNS needs to replicate? Changing minify settings in Cloudflare did not seem to happen.
  • WordPress draft posts are being cached even though page riles block wp-admin page caching.
  • Would be nice to have ad automatic Under Attack mode
  • Now all sub-domains were transferred in the setup ( id did not know for weeks)

Cloudflare status

Check out https://www.cloudflarestatus.com/ for status updates.

Don’t forget to install the CloudFlare Plugin for WordPress if you use WordPress.

More Reading

Check out my OWASP Zap and Kali Linux self-application Penetration testing posts.

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.8 host Command from the OSX CLI

v1.7 Subdomain error

v1.6 Cloudflare Attack

v1.5 WordPress Plugin

v1.4 More Reading

v1.3 added WAF snip

v1.2 Added Google Page Speed Insights and webpage rest results

v1.1 Added Y-Slow

v1.0 Initial post

Upgrading the RAM, CPU and Memory on a Vultr Ubuntu VM in the cloud

by

Upgrading the RAM, CPU and Memory on a Vultr Ubuntu VM in the cloud is quite simple.

Advertisement:



I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line.  I prefer Vultr as they are located in the country (Australia) and are easy to use.

First, you need to shut down the server from within the VM (SSH), I used the command.

Once the VM is shut down (wait a few minutes) you can turn off the VM in the Vultr GUI.

Shutdown

You can then go to Settings, Change Plan and review upgrade options.

Upgrade Options

Snapshot

Don’t forget to take a final snapshot.

Snapshot reminder

Goto the Snapshots page (read this guide to restore a snapshot) and click Take Snapshot.

Take Snapshop

You can see snapshot progress on the main screen.

Snapshot Progress

It may take a while for your snapshot to change from Pending to Processing.

Processing

Upgrade

When the snapshot is done it will auto boot and allow you to upgrade.

Manage

Choose the Upgrade specifications (Settings, Change Plan)

Upgrade Specs

Click Upgrade

Upgrade

Confirm

Confirm

The upgrade process will take a few minutes (I could see the CU and Ram was updated but the Storage was pending)

Upgrade Pending

Testing

After the upgrade happened the VM will autoboot, login and check tour specifications (Useful Linux Commands).

I use the htop command to view specification information.

I did a quick benchmark pre-optimizing and I can see a speed bump of 0.2s. Time to optimize.

Benchmark

I threw 50 concurrent clients at my website (with loader.io) and the server handled it fine with no increase above memory capacity like before.

Concurent Users

Optimize

Now I need to Optimize.  Truth be told  I did optimize and harden PHP and crashed PHP-FPM so I had o restore a VM snapshot.

Troubleshooting

If all else fails (post-upgrade configuration) you can restore the Vultr VM from a snapshot.

I hope this guide helps someone.

 

P.S If you don’t have a VM on Vultr click this link to set one up in minutes (setup guide here).

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.0 Initial post

How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic

by

This blog post will show you how to setup PHP FPM on-demand child workers in PHP 7.x to increase website traffic.

Advertisement:



My blog was experiencing a number of slow page loads and often running “sudo service php7.0-fpm restart” would resolve the problem.  I have blogged before about setting up Ubuntu Servers on AWS, Digital Ocean and Vultr but this post is about debugging and speeding up PHP on Ubuntu self-managed servers.

Background

I tried the normal tweaks in “/etc/php/7.0/fpm/php.ini” like

memory_limit = 512M

I setup servers like this.

Temporary Fix

I had even set up a temporary NGINX and php7.0-fpm restart ever 5 and 1-minute respectively until I had time to look into this.

Debug

I checked out the PHP7.0-fpm.log and I found the following

Setting up a PHP-FPM pool

Read the official guide here on configuring PHP FPM pools etc.

I edited “/etc/php/7.0/fpm/pool.d/www.conf” and added the following to set up a pool of PHP-FPM servers.

You may need more or fewer child processes depending on your needs and free memory.

After editing the PHP-FPM config file restart PHP-FPM

Restart Nginx

You will be able to view the PHP child process status by typing the following

You can use htop (commands here) to see child PHP processes in the pool and to verify free memory.

php-pool

This command is good for watching free memory on a server

I prefer to use up free memory (if available) and leave about 100mb free.

 

Hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.0 Initial Post

Using the free Adminer GUI for MySQL on your website

by

Adminer is a free GUI tool that can you can easily install on a PHP web server. Adminer allows you to easily connect to your MySQL instance, create databases/tables/indexes/rows and backup/import databases and much more.

Advertisement:


You can read my other posts on Useful Linux Terminal Commands and Useful OSX Terminal Commands.

I used to use phpMyAdmin to manage MySQL databases on AWS, Digital Ocean and Vultr but switched to Adminer due to forgotten issues.  You can always manage MySQL via command line but that is quite boring.

The below screenshots were taken on my local Development Mac Laptop (with optional OSX Apache SSL Setup (that reports “Not Secure” (but it is good enough to use locally)). I prefer to code in SSL and warn when SSL is not detected.

Downloading and Installing Adminer

Navigate to https://www.adminer.org/ and click Download.

Adminer GUIClick English only (.php file)

Adminer

Save the Adminder for MySQL (.php) file to your web server and give it a random name and put in a folder also with a random name (I use https://www.grc.com/passwords.htm to generate strong password).

Tip: Uploading this file to a live serve offers hackers and unauthorized people potential access to your MySQL server.  I would remove this file from live serves when you are not using it not to be sure.

Tip: Read my guide here on setting up NGINX, MySQL and PHP here.  Basically I did this to setup MySQL on Ubuntu 16.04.

TIP: Ensure MySQL is secure and has a good root password, also consider setting up Ubuntu Firewalls and Securing Ubuntu. Also ensure the Server is patched and does not have exploits like Spectre and meltdown.

Now you can access your Admirer php file on your Web Server (hopefully with an obfiscated name).

Randomize

Login to Adminer with your MySQL root password.

Login

Click Create databaase

Create Database

Give the database a name and choose the character coding standard (e.g UTF8 ceneral ci). Different standards have different performance impacts too.

Save

Now that you have a database you can create a table.

Adminer

Consider adding an auto-incrementing ID and say a Key and Value varchar column.

Adminer

When the table is created you can add a row to the table.

Adminer

I created one with a “TestKey” and “TestValue” row.

Adminer

The row was inserted.

Adminer

The final thing to do is add a database user that code can connect to the database with. Click Privileges.

Adminer

Click Create user

Adminer

Tick All privileges and click Save

Adminer

Now the user is added to the database

Adminer

Let’s create a PHP file and talk to the database. Let’s use parameterized queries

<?php

date_default_timezone_set('Australia/Sydney');
echo "Last modified: " . date ("F d Y H:i:s.", getlastmod()) . "<br /><br />";

// Turn on if you need to see errors
// error_reporting(E_ALL);
// ini_set('display_errors', 0);

$dbhost = '127.0.0.1';
$dbname = 'dbtest';
$dbusername = 'dbtestuser';
$dbpassword = '*****************************************'';

$con = mysqli_connect($dbhost, $dbusername, $dbpassword, $dbname);
 
// Turn on debug stuff if you need it
// echo var_dump($con);
// printf(" - Error: %s.n", $stmt->error);
 
if($con->connect_errno > 0){

    printf(" - Error: %s.n", $stmt->error);
    die("Error: Unable to connect to MySQL");

} else {

    echo "Charset set to utf8<br />";
    mysqli_set_charset($con,"utf8");
}
 
if (!$con) {

    echo "Error: Unable to connect to MySQL (E002)" . PHP_EOL;
    echo "Debugging errno: " . mysqli_connect_errno() . PHP_EOL;
    echo "Debugging error: " . mysqli_connect_error() . PHP_EOL;
    exit;

} else {

    echo "Database Connection OK<br />";
 
    echo "&nbsp; Success: A proper connection to MySQL was made! The $dbname database is great." . PHP_EOL . "<br />";
    echo "&nbsp; &nbsp;- Host information: " . mysqli_get_host_info($con) . PHP_EOL . "<br />";
    echo "&nbsp; &nbsp;- Server Info: '" . mysqli_get_server_info($con) . "'<br />";
    echo "&nbsp; &nbsp;- Server Protocol Info : ". mysqli_get_proto_info($con) . "<br />";
    echo "&nbsp; &nbsp;- Server Version: " . mysqli_get_server_version($con) . "<br />";
    //echo " - Server Connection Stats: " . print_r(vmysqli_get_connection_stats($con)) . "<br />";
    echo "&nbsp; &nbsp;- Client Version: " . mysqli_get_client_version($con) . "<br />";
    echo "&nbsp; &nbsp;- Client Info: '" . mysqli_get_client_info() . "'<br />";
 
    echo "Ready to Query the database '$dbname'.<br />";
 
    // Input Var's that are parameterized/bound into the query statement
    $in_key = mysqli_real_escape_string($con, 'TestKey');
 
    // Output Var's that the query fills after querying the database
    // These variables will be filled with data from the current returned row
    $out_id = "";
    $out_key = "";
    $out_value = "";
 
    echo "1. About to query the database: '$dbname'<br />";
    $stmt = mysqli_stmt_init($con);

    $sql = "SELECT testid, testkey, testvalue FROM tbtest WHERE testkey = ?";
    echo "SQL: $sql (In = $in_key)<br /";

    if (mysqli_stmt_prepare($stmt, $sql)) {

            echo "2. Query Returned<br />";
            /*
                Type specification chars
                Character   Description
                i   corresponding variable has type integer
                d   corresponding variable has type double
                s   corresponding variable has type string
                b   corresponding variable is a blob and will be sent in packets
            */
            mysqli_stmt_bind_param($stmt, 's', $in_key);
            mysqli_stmt_execute($stmt);
            mysqli_stmt_bind_result($stmt, $out_id, $out_key, $out_value);
            mysqli_stmt_fetch($stmt);
     
            // Do something with the 1st returned row        
            echo " - Row: ID: $out_id, KEY: $out_key, VAL: $out_value <br />";//

            // Do we have more rows to process
            while($stmt->fetch()) { 
                
                    // Output returned values
                    echo " - Row: ID: $out_id, KEY: $out_key, VAL: $out_value <br />";//
            
            }
            mysqli_stmt_close($stmt);
            
            echo "Done<br />";
        
        } else {
        
            echo "3. Error Querying<br/>";
            printf(" - Error: %s.n", $stmt->error);
        
        }
}    
?>

Result

Adminer Results

If you don’t have a server check out my guides on AWS, Digital Ocean and Vultr.

Happy coding and I hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.0 Initial Version

Adding HTTPS to Apache on OSX High Sierra

by

This guide will help you create and install a self-signed SSL certificate in Apache on OSX (HIgh Sierra) to aid local SSL/HTTPS development.

Advertisement:



UPDATE: Nov 21st 2018 – This works on OSX Mojave too.

I usually force HTTPS traffic on everything I develop (see code below in PHP).  This PHP code will direct all HTTP requests to HTTPS.

Also, you can deny non-https traffic in NGINX with online servers by editing your/etc/nginx/sites-available/default file

Add this to your Nginx sites available file (above) to force SSL at the web server.

You can also deny port 80 connections in your firewall and NGINX if you don’t trust the directive(s) above.

Apache Configuration (sor https)

Edit httpd.conf

Uncomment lines with these text strings in httpd.conf

"socache_shmcb_module" (or "LoadModule socache_shmcb_module libexec/apache2/mod_socache_shmcb.so")
"ssl_module" (or "LoadModule ssl_module libexec/apache2/mod_ssl.so")

I had to find my and “httpd.conf” file

Find your “httpd-ssl.conf” file

Copy the config file (just in case a future update overwrites it).

Now go back and edit the “httpd.conf” file

Add the following line (to the end of the file, on a new line) so the new SSL config file loads.

You can have a look at the file “httpd-ssl-localhost.conf”. Take note of paths in the VirtualHost node.

Generating a Key and Certificate

Generate a Key (I prefer filenames start with “localhost” so future updates to not overwrite it).

Check that file files ” localhost.crt” and ” localhost.key” were outputted to “/private/etc/apache2/”

pwd
/private/etc/apache2
192-168-1-200:apache2 simon$ ls -al
total 208
drwxr-xr-x   13 root  wheel    416 18 Dec 00:33 .
drwxr-xr-x  128 root  wheel   4096 16 Dec 19:41 ..
...
-rw-r--r--    1 root  wheel   1318 18 Dec 00:33 localhost.crt
-rw-r--r--    1 root  wheel   1708 18 Dec 00:33 localhost.key
...

Open “/private/etc/apache2/original/extra/httpd-ssl-localhost.conf” and replace the following text.

Replace the following text:

  • “server.crt” with “localhost.crt”
  • “server.key” with “localhost.key”

Changes shoud have been made to the following values in “<VirtualHost _default_:443>”

  • SSLCertificateFile “/private/etc/apache2/localhost.crt”

  • SSLCertificateKeyFile “/private/etc/apache2/localhost.key”

Check your apache conf

I had the following error

OSX SSL

I removed the invalid first line (I must have inserted something while searching using nano (I must have presses Option+W instead of Control+W to find in Nano)).

I ran the Apache config check again and received the following error.

I added my domain but with a 2 on the end for later use via local DNS.

<VirtualHost _default_:443>
...
ServerName https://www.mydomainname2.com:443
...

TIP: This will not by default allow you to load this address locally without, not without DNS changes (to be added soon).

Restart Apache (you may still receive a warning about the ServerName (I Ignored it)).

Restart Apache

Loading Your Site

You can now load the website localhost in your browser (it will show as insecure).

Insecure localhost

Click “Proceed to localhost (unsafe)”

The reason for no trust is the self-signed cert is not trusted by OSX or browsers.

Cert Details

OSX Certificate Trust (Key Chain Access)

Let’s tell OSX we trust this certificate (by adding it to the keychain)

Restart Apache

TIP: Check your certificate date in your browser, and clear your cache if older certificates are loading from cache.

Results

Great, Safari uses the SSL cert (and obeys Keychain’s Trust)

Safari

Google Chrome reports the Cert has issues (even though it is trusted locally). I will investigate and update this post soon.

Chrome

I suspect that Chrome will need to trust this cert fully to allow AJAX and API calls to be made.It’ss weird that Google Chrome has detected it is trusted by all users but does not trust it.

This is my next link to research a solution.

More

Read Useful OSX Terminal Commands or Useful OSX Linux Commands and Securing an Ubuntu VM with a free LetsEncrypt SSL certificate in 1 Minute first.

Hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.4 Nov 208 – Works on Mojave

v1.3 Small Edits, Added link to investigate Chrome issue (Current Version)

v1.2 Added More Explanations

v1.1 Reworded

v1.0 Initial Version

Restoring lost files on a Windows FAT, FAT32, NTFS or Linux EXT, Linux XFS volume with iRecover from diydatarecovery.nl

by

Below is my quick guide to showing how I recover lost files on a removable (or internal drive) with software on Windows. This is not a paid advert but this is what works for me (and has worked with everything I have thrown at it). Tell me what works for you (especially on OSX).

Advertisement:


“Helping one person might not change the world but it could change the world for one person” – Buddha.

I hope this guide helps someone, I have never seen someone happier than when I have been able to restore lost files or photos for someone. Not everyone backs up data (read my guide here).

Before you start

Don’t touch (format, access or install software) onto drives and USB drive that you want to restore files from or you may overwrite files on that drive. Doing this will potentially block the ability to restore files. Electronics die, software gets corrupted, this can be complex so If in doubt seek professional help and advice before proceeding. This is advice only and it’s your call with your data, Restoring files on damaged magnetic or USB drives can damage the data storage platter or storage chips, so proceed at your own risk. (legal disclaimer over, good luck).

More on how file systems and storage work

How File Systems Work

How the File Allocation Table Works

Installing DIY Data Recovery

The DIY Data Recovery software can restore images (photos) from a UBS, SD card or files (images or non-images) from data or operating system drives. If you have a system or data drive from a PC you can plug it into another PC running DIY Dat Recovery and scan and restore from the possibly bad drive to a good host drive, never restore files to a bad drive from a bad drive. I am an Apple Mac user now but only know of this Windows method for restoring files, please let me know of an Apple Mac way to restore files on FAT, NTFS and other file systems please.

I usually connect USB drives directly or connect SATA drives via a small caddy

On a working Windows computer install DIY Data Recovery software from here.

Install

Plug in your dead USB or another drive to restore files from (do not interact with the drive once Windows can see it).

Insert Drive

Do not format or modify the drive if prompted (cancel this screen).

Dont Format

Sometimes Windows will offer to fix it (do not interact with this dialog).

From memory, you can follow the steps below on a free trial version but you will need a paid version (serial number) to actually restore files. The free trial will show files that it can restore though. I paid about $60 for the software a few years ago, it is currently 59 EUROS. Enter your serial number if you have one.

Register iRecovery

After you register it (or continue with the free trial) select the drive to scan for files and click Next.

Select Drive

Here is another unreadable SATA drive

DIY Data Recovery will scan each sector and try and find files. On a smaller healthy drive, the scan won’t take long, on a larger drive (e.g a 2TB NTFS drive it may take 18+ hours).

In this case, we can see the USB drive has two bad sectors (red squares) at the front of the USB stick in the file system sectors (green squares). This possibly happened when the USB was pulled out when the operating system was writing to the USB stick. If the USB was dying the bad sectors would be at random positions.

Data fragments are listed as blue squares.

Let it scan

Here is another sample scan results

When the scan is done click ‘Save” to save the found sectors state (saving the state can save you time later).

Disclaimer: Clicking save does not save lost files it just saves the current scan state.

choose Files to recover

You can click the legend button to see the state of each file (Not processed, Presumably valid, Invalid, bad sectors etc)

Tick the files you want to be restored.

Legend

FYI: You will see some files you know, some files you forgot and some weird files. Restoring files is hit and miss and you may never see files again (do backup). Benjamin Franklin once said “Failing to plan is planning to fail”, You can also say “Failing to backup is guaranteeing lost data”.

Restoring Files

Do

  • Create a folder on your working Desktop to restore files to.
  • Do increase Read and Write cache
  • Do set longer Timeouts (e.g 1000, 5000, 10000, 60000)

Restore

Click “Start copying the selected files” to restore.

restore

Hopefully, you will have files restored, if not seek professional help. I’d recommend keeping the corrupt or source drive in a drawer, you may be able to use other tools to restore more files at a later date.

Restored Files

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

V1.1 Added more screenshot from a second DATA drive scan

v1.0 Initial Post

Building your first app on Heroku

by

Below is my blog post on setting up a hello world app on Heroku that is powered by a database (e.g Client requests a page, the page reads a database and return hello world).

Advertisement:


I found IBM/Cloudant would cost multiple thousands a month to run a good NoSQL database backend, I moved to a self-managed server on Digital Ocean in Singapore but the distance of the server killed latency and maximum throughput in Australia.  I moved to a dedicated AWS EC2 instance with a  Mongo DB cluster on AWS in Australia and it was great but pricey. A self-managed server (on Digital Ocean or Vultr) allows you to install whatever you want (e.g PHP, MySQL, MongoDB, Redis, SSL Certificate, Mail or WordPress) and configure what you want (but at what cost?).

Heroku say: “Get straight to building apps – Setting up, operating and maintaining your own platform is not where the race is won. Avoid the risk and complexity, and dedicate your energy to what really matters: building great apps.”.

heroku

Having setup serves on IBM Cloudant, Digital Ocean, AWS and Vultr (and even managing with RunCoud and Webmin) and not focusing on a self-managed server-issues like security, backup, security audits, upgrading PHP, updating ssl, setting up distributed mysql, updating https certificates etc sounds like the holy grail.

Let’s put that to the tets and build a Hello World App.

About Heroku

Heroku have a page here about what they offer.

Heroku

Sign up for Free: https://signup.heroku.com

Signup

Programming Languages (Ruby, PHP, Python, NodeJS, Java, Cloujure, Scala, Go etc). I so wanted to select “I’m not a developer” but I won’t.  I selected NodeJS.

Programming Language

Next: Verify your email, enter a password and you’ll find yourself here.

Setup Done

Now we can create an app?

App Help

Help

Ok, let’s skim the NodeJS help before we start: https://devcenter.heroku.com/articles/getting-started-with-nodejs#set-up

Let’s download the Command Line tools

Heroku CLI

Installing Heroku CLI on OSX was uneventful

Heroku CLI

 

I clicked the “I have installed the Heroku CLI” button.

Heroku wanted me to clone a repo but I logged in to Heroku the Heroku CLI and created a local Heroku folder first (in my local Terminal).

I checked the Heroku Version

I checked Heroku Help

heroku --help
Usage: heroku COMMAND

Help topics, type heroku help TOPIC for more details:

 access          manage user access to apps
 addons          tools and services for developing, extending, and operating your app
 apps            manage apps
 auth            heroku authentication
 authorizations  OAuth authorizations
 buildpacks      manage the buildpacks for an app
 certs           a topic for the ssl plugin
 ci              run an application test suite on Heroku
 clients         OAuth clients on the platform
 config          manage app config vars
 container       Use containers to build and deploy Heroku apps
 domains         manage the domains for an app
 drains          list all log drains
 features        manage optional features
 git             manage local git repository for app
 keys            manage ssh keys
 labs            experimental features
 local           run heroku app locally
 logs            display recent log output
 maintenance     manage maintenance mode for an app
 members         manage organization members
 notifications   display notifications
 orgs            manage organizations
 pg              manage postgresql databases
 pipelines       manage collections of apps in pipelines
 plugins         manage plugins
 ps              manage dynos (dynos, workers)
 redis           manage heroku redis instances
 regions         list available regions
 releases        manage app releases
 run             run a one-off process inside a Heroku dyno
 sessions        OAuth sessions
 spaces          manage heroku private spaces
 status          status of the Heroku platform
 teams           manage teams
 update          update CLI
 webhooks        setup HTTP notifications of app activity

I ran the advised commands

At this point, I followed the steps at https://devcenter.heroku.com/articles/getting-started-with-nodejs#deploy-the-app

Ran

The website is up

App Up

Where is this app located

Traceroute times out but appears to be in the eastern United States (AWS).

Q1) Can I change to Sydney Australia?

Twitter

Waiting for an answer from Twitter.

Virginia is a long way away.

Let’s test the SSL

It’s all good apart from on weak cipher (“TLS_RSA_WITH_3DES_EDE_CBC_SHA (0xa) WEAK (112)“).

SSL

A shodan scan of the IP reveals an open port 80 and 443 (as expected, nothing out of the ordinary)

Pushing Changes

I ran this to push the app.

I ran this to ensure the app was running

I ran this to open the app

This opened the apps URL in a browser.

A nice help page was pushed to my app.

I changed the contents of “node-js-getting-started/views/pages/index.ejs” and repushed.

It reported “Everything up-to-date“???

Repush

I tried to force a push (just in-case something on the server was changed)

???

Ok, let’s pull and see what’s new?

I edited “./index.js” and made the entry point “pages/index2.ejs” and tried to push and no luck.

I tried running push under sudo too (no luck)

Googling revealed I can go to https://dashboard.heroku.com/apps/YOUR-APP/deploy/heroku-gi and see what the issue is.

I logged into the app and could not find how to delete or edit it?

How do I edit files

At this point, I would have just ssh’ed in and restarted the node process in pm2 or node on my own self-managed server and edited remote files locally in sublime on my local machine.

Personally, I like to know where files live and not delve into another designers management interface.

I’ll come back to this guide later until then I’ll stick with Digital Ocean or Vultr

Linking to your own Domain on Heroku

It looks like I can use custom domains via Namecheap to link a domain to a heroku app. 

Or you can setup a domain sub domain to point to a heroku app with a DNS C Name record. 

Database Setup

Todo.. 

Node talking to the database. 

Todo.. 

App Cost on Heroku

todo: After Hello World is deployed. Digital Ocean (is as low as $5 a month) or Vultr (is as low as $2.5 a month)

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.4 added custom domain, database, CNAME info

etc

Connect to a remote server with ssh keys generated on OSX

by

Below is the way I connect to a remote server via SSH keys generated on OSX.

Advertisement:


Setting up a server

When you set up an Ubuntu server on Vultr (read my guide on setting up a Vultr server for as low as $2.5 a month) or Digital Ocean (use this link to get two months free when you setup an Ubuntu server on digital ocean) you can specify an SSH key to use for remote connections during the server create stage (old guide here).

How to create an SSH key on OSX to use to connect to a remote server

Run the following command (“sudo ssh-keygen -t rsa“) to generate an ssh key paid in “~/.ssh/” on OSX.

TIP: It is a good idea to also generate a passphrase to use with the key (double protection). You will be prompted to enter this password to use the RSA key.

You can now see the generated keys in ~/.ssh/

You can view the contents of the public file (you can use this when generating Digital Ocean, Vultr, AWS or Azure or other cloud servers).

fyi: Replace 123.123.123.123 with your remote serves ip.

How to connect to a server (the old way).

As long as your host added the desired public ssh key file contents to the server (adding the public ssh key contents to “~/.ssh/authorized_keys” you will be able to connect to the server.

Run the following command on OSX command line to connect to the server via SSH.

You should see..

Congratulations, you should now be able to connect to your server via SSH.

Securing your ubuntu Server

Read my guides here, here and here.

Don’t forget to add a firewall and set up an SSL certificate.

How to connect to a server (faster way).

todo: ~/.ssh/config method

Now we can connect to your remote server with the shorter method.

todo: ~/.ssh/config method

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.0 Initial Post

etc

Updating PHP 7.0 to 7.1 on an Ubuntu 16.04 Vultr VM

by

Here is how you can quickly update PHP 7.0 to 7.1 on a Vultr Ubuntu domain.

Advertisement:



I have configured a number of Vultr domains with NGINX and PHP 7.1 FPM and today I realised I need to update PHP 7.0 to 7.1 to fix a  few security exploits (read more here and here on securing Ubuntu in the cloud). PHP has a good page where you can keep up to date with PHP news here https://secure.php.net/. You can also view the PHP bug tracker to view bugs here. PHP aggregation user @php_net on twitter is good to follow, the official PHP twitter account is @official_php.

I have not noticed in daily Ubuntu package updates no option to update PHP 7.0 to 7.1, I must have to update manually.

WARNING: Backup your site and test this on a non production server before doing it on a live server.  I had an issue with PHP 7.1 breaking WordPress 3.9 (mysql issues with some plugins) and I had to roll back to 7.0 (see rollback tips in troubleshooting below). WordPress says it is PHP 7.1 compatible but issues exist. WordPress 3.9 ditches “mysql” and used “mysqli” and when instead PHP 7.1 WordPress could not find “mysqli”?

List packages with updates

You can run the following to view upgradable packages (TIP: Backup NGINX and other configuration files before any upgrades).

Update your server packages

Reboot

You should now see this on startup

You can view your installed PHP configuration file and installed version by typing to following in your servers command line.

Now let’s install a package viewer

Search installed packages (or non-installed) PHP packages.

Uninstall all local PHP related packages

Confirm packages are uninstalled

Install PHP 7.1 and common packages

Verify PHP 7.1 installation

Reboot

See if the PHP 7.1 FPM service has started

sudo systemctl | grep php
> php7.1-fpm.service

Restart PHP 7.1 FPM Service

Edit your /etc/nginx/sites-enabled/default and change the fastcgi_pass from “7.0” to “7.1”

Edits:

Reload NGINX configuration and restart NGINX

Your website should now be back up and running PHP 7.1

PHP 7.1

Post Install Tasks

View this blog post on other useful linux commands.

Run a Lynis security scan.

Edit your PHP.ini file and add required changes (e.g upload sizes).

sudo nano /etc/php/7.1/fpm/php.ini
# upload_max_filesize = 2M
+ upload_max_filesize = 8M

Troubleshooting

View PHP configuration values (add this to a debug.php and load in in a browser)

<?php

// Show all information, defaults to INFO_ALL
phpinfo();

// Show just the module information.
// phpinfo(8) yields identical results.
phpinfo(INFO_MODULES);

?>

I broke my WordPress 3.9 when I tried to update to PHP 7.1 so I rolled back to 7.0.

Google help had me stuck for a while when I had issues purging php 7.1.

Purge Error

Because my blog (with install steps) was down I used this site to help be find the commands to run.

Conclusion

Sometimes going with cutting edge tech you will go out on a limb, ensure you know and can restore a working site if need be.

Always have a backup and restore plan.

Hope this guide helps.

Donate and make this blog better


Ask a question or recommend an article
[contact-form-7 404 "Not Found"]

v1.35 WordPress 3.9 error with PHP 7.1

Setting up additional server storage on cloud servers (block storage on Vultr)

by

Vultr has a generous disk quota with the cloud servers you can set up. But what do you do when you want more space than the default allocation (for backup or application data)?

Advertisement:



I have blogged before about setting up an Ubuntu server on the cloud on Vultr and configuring it if you do not already have a cloud server.

Vultr allows you to set up a server in minutes.

Server

A Vultr $2.5 a month server comes with 20GB storage, a $20 a month server comes with 60GB of SSD storage.

Vultr does offer more storage for about 0.10c per GB. At this time or writing Vultr allows you to add more storage to serves in NY/NJ (only). Read my guide on moving data between servers with RSync. And cond forget yo secure your server with a free SSL certificate and secure it (read more here and here).

An additional 10GB of storage would cost $1/m.

10GB

An additional 50GB of storage would cost $5/m.

50GB

An additional 100GB of storage would cost $10/m.

100GB

An additional 250 GB of storage would cost $25/m

250GB

View the Vultr pricing calculator here. Vultr does say that you can resize your block storage volume but there are manual actions and risks involved so get the space you need early on and prevent resizing later.

Read the Vultr Block Storage FAQ here: https://www.vultr.com/docs/block-storage

Vultr did offer early customers in (limited location’s) a free 50GB storage (read more on these limits here).

I am going to spin up a Block storage and attach to my server in Sydney.

fyi: Read the official guide on Attacking Block Storage to a Vultr server.

1. Login to your Vultr admin panel ( https://my.vultr.com/ ) and click Block Storage (  https://my.vultr.com/blockstorage/ ).

2. Click Add Block Storage

Add Block Storage

3. Choose the size of your block storage volume.

New Block Storage

Darn, I can’t choose Syndey yet as a location to create a block storage volume (I have asked Vultr when we can) so I’ll continue this guide with my existing (free) 50GB volume in New Jersey) and mount it in a server in NY/NJ (and also Syndey).

It appears I can’t connect to a  Block Storage volume outside the block storages location (data centre).

Manage Block Storage

 

You will need to attach the block storage volume to the server at that data centre location or you will get this error when you try and connect to it later.

Error

 

In my case, the server did not automatically restart so I manually restarted it.

 

Connecting the Block Storage to your VM

From the Vultr admin panel ( https://my.vultr.com ), Block Storage ( https://my.vultr.com/blockstorage/ ) you can manage individual Block Storage volumes and see the mounting information.

e.g

FYI: You can only connect to a  block storage from the same location (one server at a time I’d imagine).

4. From he Vultr Admin panel SSH into the server (in the same location).  See my guide here on setting up a Vultr server and configuring it.

Vultr say’s “Block storage is connected to your server as /dev/vdb. We do not create any filesystems on it by default.” Official Block storage documentation is located here.

5. Run the commands listed in the Block Storage screen (above)

Mount

Error: In my case, the echo command failed  add to configuration to the /etc/fstab file (even with sudo) and the mount command failed?

I checked the /etc/fstab file contents

I manually edited the /etc/fstab file and added the mount point configuration as suggested by Vultr.

Contents

6. I re-ran the mount command

sudo mount /mnt/blockstorage
#

Success

I can now directory list in the block storage volume.

ls /dev/vdb1 -al
brw-rw---- 1 username disk 253, 17 Nov  7 21:18 /dev/vdb1

 

Now let’s attach it to another folder in the root folder (e.g /data)

First, unmount the volume

Edit the /etc/fstab file with sudo nano

Change the mount point somewhere else (e.g /data)

Make a folder in the new path (/data), If you don’t do this the mount will fail.

Remount the volume (but use the new path)

sudo mount /data

You can now use the path and new storage.

cd /data
mkdir /data/test
cd /data/test
pwd
# /data/test/
sudo nano /data/test/test.txt

Nice

 

Disposing of Block Storage

TIP: Move or backup any data before you destroy or detach the volume.

First, you will need to unmount the volume (SSH session with your server).

Then remove the entry from the /etc/fstab file

Then you can navigate to the https://my.vultr.com/blockstorage/ and edit the said block storage volume and detach the volume (this will cause the server to reboot).

Detatch

After a few minutes you can delete the volume from the edited Block Storage Volume page  (click the Trashcan up the upper right).

Detatch

Done, You can now add and remove Block Storage volumes on Vultr.

 

How to check the disk usage of the block storage volume

You may need to remind yourself of the block storage volume (cat the /etc/fstab file and view the drive information on the mount line).

cat /etc/fstab
# .. /dev/vdb1 ..

How much space is used/free

df -h /dev/vdb1
Filesystem      Size  Used Avail Use% Mounted on
/dev/vdb1        50G   52M   47G   1% /data

You can also show the usage information in that mounted folder

du -xsch /data
10G     /folder1
10G     /folder2
20G     total

Use the pydf tool to view mounted partitions

Install pydf

Use pydf

 

Troubleshooting

  • You need to attach the block storage volume and reboot in the Vultr admin panel before mounting.
  • The echo command (as documented by Vultr) may not add information to the /etc/fstab file (a manual edit will work).

 

 

How to Resize a Block Storage Volume on Vultr.

Coming soon (if requested below).

 

Donate and make this blog better


Ask a question or recommend an article
[contact-form-7 404 "Not Found"]

v1.2 added disk usage information