Here is how you can update to PHP 7.2.12 on Ubuntu. PHP has a support page that declares the support date ranges and support types: http://php.net/supported-versions.php
How
How to install PHP 7.2.latest on Ubuntu 16.04
How to install PHP 7.2.latest on Ubuntu 16.04/ Ubuntu 18.04/Debian etc/
Advertisement:
I have a number of guides on moving away from CPanel, Setting up VM’s on UpCloud, AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. PHP is my programming language of choice.
PHP has a support page that declares the support date ranges and support types: http://php.net/supported-versions.php
A version of PHP is either actively supported, security fix supported or end of life. Read this post to check WordPress for PHP compatibility.
From time to time vulnerabilities come up that require PHP updates to be applied.
Multiple flaw found in #PHP, most severe of which could allow arbitrary code execution
Affected Versions:
PHP 7.2 —prior to 7.2.5
PHP 7.1 —prior to 7.1.17
PHP 7.0 —prior to 7.0.30
PHP 5.0 —prior to 5.6.36https://t.co/TtiqXePoHuUpgrade to the latest version of PHP immediately
— The Hacker News (@TheHackersNews) May 1, 2018
#PHP 7.2.12 has been released https://t.co/iNXGYTs0PX
— Neustradamus (@neustradamus) November 9, 2018
Source Link here
Advertisement:
I have guides on setting up PHP 7 here on Digital Ocean, here on AWS and here on Vultr. I have tried upgrading to PHP 7.1 in the past with no luck (I forgot to change something and rolled back to 7.0).
FYI: I have a guide on setting up PHP child workers so the output from some commands below may be different than yours. Here are the steps I performed to install PHP 7.2 alongside 7.0 then switch. to 7.2.
Backup your system
Do perform a Snapshot or Backup before proceeding. Nothing beats a quick restore if things fail.
Note: Use this information at your own risk.
Updating php 7.2.12 to 7.2.12
Update your Ubuntu systems
1 | apt-get update && apt-get upgrade |
Updating from an older php (e.g 5.x, 7.1, 7.1 to say 7.2.12)
Backup PHP
1 2 | cd /etc/php zip -r php7.0backup.zip 7.0/ |
Install Helper
This software provides an abstraction of the used apt repositories. It allows you to easily manage your distribution and independent software vendor software sources. More Info
1 | apt-get install python-software-properties |
Add the main PHP repo (more information)
1 | add-apt-repository ppa:ondrej/php |
Update the package lists
“In a nutshell, apt-get update doesn’t actually install new versions of the software. Instead, it updates the package lists for upgrades for packages that need upgrading, as well as new packages that have just come to the repositories.” from here
1 | apt-get update |
List Installed Packages (optional)
1 | dpkg -l |
Install PHP 7.2
1 | apt-get install php7.2 |
Install common PHP modules
1 | apt-get install php-pear php7.2-curl php7.2-dev php7.2-mbstring php7.2-zip php7.2-mysql php7.2-xml |
Install PHP FPM
1 | apt-get install php7.2-fpm |
Update all packages (may be needed to update from php 7.2.4 to 7.2.5)
1 | sudo apt-get upgrade |
Edit your NGINX sites-available config
1 2 | sudo nano /etc/nginx/sites-available/default # I set: fastcgi_pass /run/php/php7.2-fpm.sock; |
Edit your NGINX sites-enabled config
1 2 | sudo nano /etc/nginx/sites-enabled/default # I set: fastcgi_pass unix:/var/run/php/php7.2-fpm.sock; |
I edited these lines
1 2 3 4 5 | location ~ \.php$ { ... fastcgi_pass unix:/var/run/php/php7.2-fpm.sock; ... } |
Edit your PHP config (and make desired changes)
1 | sudo nano /etc/php/7.2/fpm/php.ini |
Edit your PHP pool config file (as required). See this guide here.
e.g.
> cgi.fix_pathinfo=0
> max_input_vars = 1000
> memory_limit = 1024M
> max_file_uploads = 8M
> post_max_size = 8M
1 | sudo nano /etc/php/7.2/fpm/pool.d/www.conf |
Make sure you set: listen = /run/php/php7.2-fpm.sock
Advertisement:
Set PHP 7.2 as the default PHP
1 | update-alternatives --set php /usr/bin/php7.2 |
Check your PHP version
1 | php -v |
Reload PHP
1 | sudo service php7.2-fpm reload |
Reload NGINX
1 2 3 | nginx -t nginx -s reload /etc/init.d/nginx restart |
Check the status of your PHP (and child workers)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | sudo service php7.2-fpm status ● php7.2-fpm.service - The PHP 7.2 FastCGI Process Manager Loaded: loaded (/lib/systemd/system/php7.2-fpm.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2018-05-04 19:02:27 AEST; Docs: man:php-fpm7.2(8) Process: 123456 ExecReload=/bin/kill -USR2 $MAINPID (code=exited, status=0/SUCCESS) Main PID: 123456 (php-fpm7.2) Status: "Processes active: 0, idle: 10, Requests: 0, slow: 0, Traffic: 0req/sec" Tasks: 11 Memory: 30.5M CPU: 10.678s CGroup: /system.slice/php7.2-fpm.service ├─16494 php-fpm: master process (/etc/php/7.2/fpm/php-fpm.conf) ├─16497 php-fpm: pool www ├─16498 php-fpm: pool www ├─16499 php-fpm: pool www ├─16500 php-fpm: pool www ├─16501 php-fpm: pool www ├─16502 php-fpm: pool www ├─16503 php-fpm: pool www ├─16504 php-fpm: pool www ├─16505 php-fpm: pool www └─16506 php-fpm: pool www |
Check your website.
Advertisement:
Troubleshooting
Guides that helped me.
https://thishosting.rocks/install-php-on-ubuntu/
Check your log files
1 | tail /var/log/nginx/error.log |
Debug FPM Service
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | systemctl status php7.2-fpm.service ● php7.2-fpm.service - The PHP 7.2 FastCGI Process Manager Loaded: loaded (/lib/systemd/system/php7.2-fpm.service; enabled; vendor preset: enabled) Active: active (running) since Sun 2018-05-06 00:18:55 AEST; 7min ago Docs: man:php-fpm7.2(8) Process: 123456 ExecReload=/bin/kill -USR2 $MAINPID (code=exited, status=0/SUCCESS) Main PID: 123 (php-fpm7.2) Status: "Processes active: 0, idle: 10, Requests: 44, slow: 0, Traffic: 0req/sec" Tasks: 11 Memory: 212.6M CPU: 12.052s CGroup: /system.slice/php7.2-fpm.service ├─438 php-fpm: master process (/etc/php/7.2/fpm/php-fpm.conf) ├─441 php-fpm: pool www ├─442 php-fpm: pool www ├─443 php-fpm: pool www ├─444 php-fpm: pool www ├─445 php-fpm: pool www ├─446 php-fpm: pool www ├─447 php-fpm: pool www ├─449 php-fpm: pool www ├─450 php-fpm: pool www └─451 php-fpm: pool www May 06 00:18:55 server systemd[1]: Stopped The PHP 7.2 FastCGI Process Manager. May 06 00:18:55 server systemd[1]: Starting The PHP 7.2 FastCGI Process Manager... May 06 00:18:55 server systemd[1]: Started The PHP 7.2 FastCGI Process Manager. |
Remove PHP 7.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | sudo apt-get purge php7.0-common Reading package lists... Done Building dependency tree Reading state information... Done The following packages were automatically installed and are no longer required: libaspell15 libauthen-pam-perl libc-client2007e libio-pty-perl libmcrypt4 librecode0 libtidy-0.99-0 libxmlrpc-epi0 linux-headers-4.4.0-109 linux-headers-4.4.0-109-generic linux-headers-4.4.0-112 linux-headers-4.4.0-112-generic linux-headers-4.4.0-87 linux-headers-4.4.0-87-generic linux-headers-4.4.0-96 linux-headers-4.4.0-96-generic linux-image-4.4.0-109-generic linux-image-4.4.0-112-generic linux-image-4.4.0-87-generic linux-image-4.4.0-96-generic linux-image-extra-4.4.0-109-generic linux-image-extra-4.4.0-112-generic linux-image-extra-4.4.0-87-generic linux-image-extra-4.4.0-96-generic mlock Use 'sudo apt autoremove' to remove them. The following packages will be REMOVED: php7.0-cli* php7.0-common* php7.0-curl* php7.0-fpm* php7.0-gd* php7.0-imap* php7.0-intl* php7.0-json* php7.0-mbstring* php7.0-mcrypt* php7.0-mysql* php7.0-opcache* php7.0-pspell* php7.0-readline* php7.0-recode* php7.0-sqlite3* php7.0-tidy* php7.0-xml* php7.0-xmlrpc* php7.0-xsl* |
PHP 7.0 Removed 🙂
Remove other unused packages
1 | sudo apt autoremove |
At the time of writing (November the 18th 2018) PHP 7.2.12 is the latest version of PHP and PHP 7.3 will be out at the end of the year.
Good luck and I hope this guide helps someone
Advertisement:
Ask a question or recommend an article
Revision History
v1.4 Updated the post to mention PHP 7.0 EOL
v1.3 Updated to add PHP 7.2.12 information
v1.2 PHP 7.2.9 and PHP 7.2 updates
v1.1 Remove PHP 7.0 steps
v1.0 Initial post
How to use the UpCloud API to manage your UpCloud servers
How to use the UpCloud API to manage your UpCloud servers.
Read the full article here: https://fearby.com/article/how-to-use-the-upcloud-api-to-manage-your-upcloud-servers/
How to use the UpCloud API to manage your UpCloud servers
How to use the UpCloud API to manage your UpCloud servers.
Advertisement:
If you have not read my previous posts I have now moved my blog etc to the awesome UpCloud host. Sign up using this link to get $25 free credit.
I recently compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here).
Here is my blog post on moving from Vultr to UpCloud.
Spoiler: UpCloud performance is great.
I have never had an UpCloud page load take longer than 2 seconds since moving.
UpCloud API
UpCloud has an API that we can opt into to using where we can manage servers. Read the official UpCloud API documentation here.
The API allows you to control:
- Accounts
- Pricing
- Zones
- Timezones
- Plans
- Servers
- Storages
- IP-Addresses
- Firewall
- Tags
- etc
Create a sub-account to query the API
You should create a new user account (in the UpCloud dasbboard) just for API access. I created two accounts for use on my server and on my home laptop and my server (and set a limiting IP(s) that can access it).
Create a Sub Account for API Access
Login to your UpCloud account (create an account here and get $25 free credit),
- Click My Accounts,
- Click User Accounts,
- Click Change on your user and enable API connections.
- TIP: Set up an IP rule to limit access to your API for security (I set up a VPN to get a static IP on my dynamic IP Internet host at home)).
- Save the changes
TIP: Lock down the account to have the minimum permissions required.
e.g
- Disable access to the control panel (Untick).
- Allow API Connections (Tick) and specify an IP
- Disable access to billing contact (Untick).
- Disable access to billing section in the control panel (Untick).
- Disable allowing of emails to billing contact (Untick).
- Allow or Remove access to all server (or manually add access to desired servers)
- Allow or Remove access to modify storage (or manually allow or remove access to desired storage)
- etc
Save the account.
Now let’s make our first API call
I use OSX and I use the awesome Paw API testing tool from https://paw.cloud (This is not a plug, they are awesome). Postman is a popular API testing tool too. Any good programing language or CLI will allow you to send API requests.
First, let’s prepare the authorization string (this is a Base64 encoded combination of your username and password) read more here.
- Head over to https://www.base64encode.org/
- Click the Encode tab
- Add your “username:password” (without the quotes).
- Click Encode
A Base64 string will be outputted 🙂
e.g > eW91cmFwaXVzZXJuYW1lOnlvdXJzdXBlcnNlY3VyZXBhc3N3b3Jk
fyi
You can encode also Encode and Decode Base64 from the Ubuntu Command line
Encode Base64 from the CLI Sample
1 2 | echo -n 'yourapiusername:yoursupersecurepassword' | base64 eW91cmFwaXVzZXJuYW1lOnlvdXJzdXBlcnNlY3VyZXBhc3N3b3Jk |
Decode Base64 from the CLI Sample
1 2 | echo `echo eW91cmFwaXVzZXJuYW1lOnlvdXJzdXBlcnNlY3VyZXBhc3N3b3Jk | base64 --decode` yourapiusername:yoursupersecurepassword |
Now we can add an “Authorization Basic” token to the API request in Paw.
A quick test of the UpCloud Prices API endpoint https://api.upcloud.com/1.2/price reveals the API is working.
I can now see a full breakdown of my service prices in JSON 🙂
Query My Account
OK, Let’s see how much credit I have left by querying the https://api.upcloud.com/1.2/account, I duplicated the item in Paw and changed the URL to https://api.upcloud.com/1.2/account but no data returned?
I had to enable “Access to Billing section in Control Panel” for the user before this data returned from the API (make sense).
> HTTP/1.1 200 OK
Query (GET)
1 2 3 4 | GET /1.2/account HTTP/1.1 Host: api.upcloud.com User-Agent: Paw/3.1.7 (Macintosh; OS X/10.13.5) NSURLConnection/1452.23 Authorization: Basic ******************************************* |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 | HTTP/1.1 200 OK Date: Sun, 17 Jun 2018 04:23:32 GMT Content-Type: application/json; charset=UTF-8 Connection: close Content-Length: 91 Server: Apache { "account" : { "credits" : 2500.00, "username" : "yourapiusername" } } |
“2500.00” = cents ($25)
Query All of Your Servers
Ok, Let’s get server information by querying https://api.upcloud.com/1.2/server
Query (GET)
1 2 3 4 | GET /1.2/server HTTP/1.1 Host: api.upcloud.com User-Agent: Paw/3.1.7 (Macintosh; OS X/10.13.5) NSURLConnection/1452.23 Authorization: Basic ##############base64hash############## |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | HTTP/1.1 200 OK Date: Sun, 17 Jun 2018 04:32:22 GMT Content-Type: application/json; charset=UTF-8 Connection: close Content-Length: 1154 Server: Apache { "servers" : { "server" : [ { "core_number" : "1", "hostname" : "server1nameredacted.com", "license" : 0, "memory_amount" : "2048", "plan" : "1xCPU-2GB", "plan_ipv4_bytes" : "3472464313", "plan_ipv6_bytes" : "166293599", "state" : "started", "tags" : { "tag" : [ "tag1" ] }, "title" : "server1nameredacted.com", "uuid" : "########-####-####-####-############", "zone" : "us-chi1" }, { "core_number" : "1", "hostname" : "server2nameredacted.com", "license" : 0, "memory_amount" : "1024", "plan" : "1xCPU-1GB", "plan_ipv4_bytes" : "198911", "plan_ipv6_bytes" : "19742", "state" : "started", "tags" : { "tag" : [ "tag2" ] }, "title" : "server1nameredacted.com", "uuid" : "########-####-####-####-############", "zone" : "us-chi1" } ] } } |
Query Server Information
I have redated the UUID’s for my servers but once you know them you can query them by hitting https://api.upcloud.com/1.2/server/########-####-####-####-############
Query (GET)
1 2 3 4 | GET /1.2/server/########-####-####-####-############ HTTP/1.1 Host: api.upcloud.com User-Agent: Paw/3.1.7 (Macintosh; OS X/10.13.5) NSURLConnection/1452.23 Authorization: Basic ##############base64hash############## |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | HTTP/1.1 200 OK Date: Sun, 17 Jun 2018 04:45:14 GMT Content-Type: application/json; charset=UTF-8 Connection: close Content-Length: 1656 Server: Apache { "server" : { "boot_order" : "cdrom,disk", "core_number" : "1", "firewall" : "on", "host" : redacted, "hostname" : "server1nameredacted.com", "ip_addresses" : { "ip_address" : [ { "access" : "private", "address" : "##.#.#.###", "family" : "IPv4" }, { "access" : "public", "address" : "###.###.###.###", "family" : "IPv4", "part_of_plan" : "yes" }, { "access" : "public", "address" : "####:####:####:####:####:####:########", "family" : "IPv6" } ] }, "license" : 0, "memory_amount" : "2048", "nic_model" : "virtio", "plan" : "1xCPU-2GB", "plan_ipv4_bytes" : "3519033266", "plan_ipv6_bytes" : "168200052", "state" : "started", "storage_devices" : { "storage_device" : [ { "address" : "virtio:0", "boot_disk" : "0", "part_of_plan" : "yes", "storage" : "########-####-####-####-############", "storage_size" : 50, "storage_title" : "system", "type" : "disk" } ] }, "tags" : { "tag" : [ "fearby" ] }, "timezone" : "Australia/Sydney", "title" : "server1nameredacted.com", "uuid" : "########-####-####-####-############", "video_model" : "cirrus", "vnc" : "off", "vnc_password" : "#########################", "zone" : "us-chi1" } } |
The servers name, IPv4 and IPV6 network adapters are listed, CPU(s), Memory, Disk Sized and UUID’s are all visible 🙂
Surprisingly the VNC password is visible (enabling access to the root console).
TIP: Ensure your API account is safe and secure.
Query Storage Information
Now, Let’s query the storage with the GUID from above by querying https://api.upcloud.com/1.2/storage/########-####-####-####-############
Query (GET)
1 2 3 4 | GET /1.2/storage/########-####-####-####-############ HTTP/1.1 Host: api.upcloud.com User-Agent: Paw/3.1.7 (Macintosh; OS X/10.13.5) NSURLConnection/1452.23 Authorization: Basic ##############base64hash############## |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | HTTP/1.1 200 OK Date: Sun, 17 Jun 2018 04:53:36 GMT Content-Type: application/json; charset=UTF-8 Connection: close Content-Length: 559 Server: Apache { "storage" : { "access" : "private", "backup_rule" : {}, "backups" : { "backup" : [ "########-####-####-####-############" ] }, "license" : 0, "part_of_plan" : "yes", "servers" : { "server" : [ "########-####-####-####-############" ] }, "size" : 50, "state" : "online", "tier" : "maxiops", "title" : "system", "type" : "normal", "uuid" : "########-####-####-####-############", "zone" : "us-chi1" } } |
I can see information about the storage’s assigned server and backups 🙂
Query Backup Information
Backup storage can be queried with the same storge API endpoint https://api.upcloud.com/1.2/storage/########-####-####-####-############
Query (GET)
1 2 3 4 | GET /1.2/storage/014fd483-ea90-4055-b445-bf2011951999 HTTP/1.1 Host: api.upcloud.com User-Agent: Paw/3.1.7 (Macintosh; OS X/10.13.5) NSURLConnection/1452.23 Authorization: Basic ##############base64hash############## |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | HTTP/1.1 200 OK Date: Sun, 17 Jun 2018 05:01:11 GMT Content-Type: application/json; charset=UTF-8 Connection: close Content-Length: 412 Server: Apache { "storage" : { "access" : "private", "created" : "2018-06-16T04:47:56Z", "license" : 0, "origin" : "########-####-####-####-############", "servers" : { "server" : [] }, "size" : 50, "state" : "online", "title" : "On-Demand Backup", "type" : "backup", "uuid" : "########-####-####-####-############", "zone" : "us-chi1" } } |
Rename Backup
One thing that I would like to be able to do is to rename on-demand backups in the UpCloud dashboard (this is not a feature yet) but I can rename manual backup’s in the API though 🙂
Boring “On-Demand Backup” label.
I tried sending JSON to https://api.upcloud.com/1.2/storage/########-####-####-####-############ to rename a backup but kept getting an error?
JSON
{
> “storage”: {
> “title”: “Latest manual backup , Working NGINX, PHP, MySQL w Tweaks”,
> “size”: “50”
> }
> }
Result
> “error_code” : “CONTENT_TYPE_INVALID”,
> “error_message” : “The Content-Type header has an invalid value.”
I googled and found an old manual for UpClouds API (official support here).
I added these missing content type headers (108 was the length in chars of the payload)
1 2 | > Content-Type: application/json; Charset=UTF-8' > Content-Length: 108 |
Still no go?
I think the content length value is wrong, more here.
I fixed it, it turned out I had a semicolon in the Content-Type value. The JSON RFC always assumes that Content-Type is UTF8 encoded (more here).
This Fails
1 | Content-Type: application/json; charset=utf-8 |
This Works
1 | Content-Type: application/json |
Now I can rename my Backup (storage). I manually calculated the length of the JSON payload and added a “Content-Length” header and value.
Query (PUT)
1 2 3 4 5 6 7 8 | PUT /1.2/storage/########-####-####-####-############ HTTP/1.1 Host: api.upcloud.com User-Agent: Paw/3.1.7 (Macintosh; OS X/10.13.5) NSURLConnection/1452.23 Content-Type: application/json Content-Length: 113 Authorization: Basic ##############base64hash############## {"storage":{"size":"50","title":"Latest manual backup , Working NGINX, PHP, MySQL w Tweaks"}} |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | HTTP/1.1 202 ACCEPTED Date: Sun, 17 Jun 2018 05:47:02 GMT Content-Type: application/json; charset=UTF-8 Connection: close Content-Length: 453 Server: Apache { "storage" : { "access" : "private", "created" : "2018-06-16T04:47:56Z", "license" : 0, "origin" : "########-####-####-####-############", "servers" : { "server" : [] }, "size" : 50, "state" : "online", "title" : "Latest manual backup , Working NGINX, PHP, MySQL w Tweaks", "type" : "backup", "uuid" : "########-####-####-####-############", "zone" : "us-chi1" } } |
Success 🙂
Create a Backup
Backups can be performed with a “/backup” added to the end of the query string.
Query (POST)
1 2 3 4 5 6 7 8 9 10 11 12 | POST /1.2/storage/########-####-####-####-############/backup HTTP/1.1 Host: api.upcloud.com User-Agent: Paw/3.1.7 (Macintosh; OS X/10.13.5) NSURLConnection/1452.23 Content-Type: application/json Content-Length: 100 Authorization: Basic ##############base64hash############## { "storage": { "title": "Sunday 17th Latest backup , Working NGINX, PHP, MySQL w Tweaks" } } |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | HTTP/1.1 201 CREATED Date: Sun, 17 Jun 2018 06:17:35 GMT Content-Type: application/json; charset=UTF-8 Connection: close Content-Length: 487 Server: Apache { "storage" : { "access" : "private", "created" : "2018-06-17T06:17:35Z", "license" : 0, "origin" : "########-####-####-####-############", "progress" : "0", "servers" : { "server" : [] }, "size" : 50, "state" : "maintenance", "title" : "Sunday 17th Latest backup , Working NGINX, PHP, MySQL w Tweaks", "type" : "backup", "uuid" : "########-####-####-####-############", "zone" : "us-chi1" } } |
Success (UpCloud GUI)
Conclusion
UpCloud does have great API docs.
I can easily integrate this into bash scripts to manage my servers via API and a future Java app for managing servers.
Paw does give CURL output to allow me to copy working API’s for use in BASH 🙂
More to come
- BASH Script to Deploy and configure a server on UpCloud via Initalization scripts (or manual) (1 week)
- JAVA App to manage your server (3 months)
If you are signing up for UpCloud please consider using my referral code and get $25 credit for free.
Read my setup guide here.
https://www.upcloud.com/register/?promo=D84793
I hope this guide helps someone.
Ask a question or recommend an article
Revision History
V1.1 updated typo
v1.0 Initial Post.
Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 4 of 4
How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 4 of 4
Advertisement:
Read Part 1, Part 2, Part 3 or Part 4
I ran the MySQL benchmark preparation command again (no problem this time).
1 2 3 4 5 | sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=###################### prepare sysbench 0.4.12: multi-threaded system evaluation benchmark Creating table 'sbtest'... Creating 1000000 records in table 'sbtest'... |
Test table and records created
Now I can benchmark MySQL on my main server.
1 | sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=################################# --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run |
RAW Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 8 Doing OLTP test. Running mixed OLTP test Doing read-only test Using Special distribution (12 iterations, 1 pct of values are returned in 75 pct cases) Using "BEGIN" for starting transactions Using auto_inc on the id column Threads started! Time limit exceeded, exiting... (last message repeated 7 times) Done. OLTP test statistics: queries performed: read: 336210 write: 0 other: 48030 total: 384240 transactions: 24015 (400.09 per sec.) deadlocks: 0 (0.00 per sec.) read/write requests: 336210 (5601.24 per sec.) other operations: 48030 (800.18 per sec.) Test execution summary: total time: 60.0242s total number of events: 24015 total time taken by event execution: 480.0242 per-request statistics: min: 1.79ms avg: 19.99ms max: 141.00ms approx. 95 percentile: 37.49ms Threads fairness: events (avg/stddev): 3001.8750/27.36 execution time (avg/stddev): 60.0030/0.01 |
Results
queries performed (in 60 seconds):
- read: 336210
- other: 48030
- total: 384240
I decided to add an index to see if I can speed this query up (read the MySQL index page here). I added an index (in Adminer) on the columns “Id” and “pad” for the sbtest table in the test database
I restarted the MySQL process
1 2 | mysql restart [ ok ] Restarting mysql (via systemctl): mysql.service. |
I ran the same benchmark again.
Raw Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=########################## --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 8 Doing OLTP test. Running mixed OLTP test Doing read-only test Using Special distribution (12 iterations, 1 pct of values are returned in 75 pct cases) Using "BEGIN" for starting transactions Using auto_inc on the id column Threads started! Time limit exceeded, exiting... (last message repeated 7 times) Done. OLTP test statistics: queries performed: read: 426538 write: 0 other: 60934 total: 487472 transactions: 30467 (507.69 per sec.) deadlocks: 0 (0.00 per sec.) read/write requests: 426538 (7107.67 per sec.) other operations: 60934 (1015.38 per sec.) Test execution summary: total time: 60.0110s total number of events: 30467 total time taken by event execution: 479.9124 per-request statistics: min: 5.75ms avg: 15.75ms max: 138.57ms approx. 95 percentile: 25.10ms Threads fairness: events (avg/stddev): 3808.3750/8.70 execution time (avg/stddev): 59.9891/0.00 |
Results
The quick index added 20% extra throughput on queries 🙂
Don’t forget to delete your test database
1 | DROP DATABASE `test`; |
Viewing MySQL Index Usage (on the “test” database)
Query to show Index stats for a table ‘test’
1 2 3 4 5 6 7 8 9 10 11 12 | SELECT OBJECT_SCHEMA as 'Database', OBJECT_NAME as 'Table', INDEX_NAME as 'Index', COUNT_STAR, SUM_TIMER_WAIT, MIN_TIMER_WAIT, AVG_TIMER_WAIT, MAX_TIMER_WAIT, COUNT_READ, SUM_TIMER_READ, MIN_TIMER_READ, AVG_TIMER_READ, MAX_TIMER_READ, COUNT_FETCH, SUM_TIMER_FETCH, MIN_TIMER_FETCH, AVG_TIMER_FETCH, MAX_TIMER_FETCH FROM performance_schema.table_io_waits_summary_by_index_usage WHERE object_schema = 'test' |
I can see the MySQL PRIMARY index is getting used 🙂
Read more in viewable query stats (columns) here.
Other System Information Tools
Show processor information
1 | cat /proc/cpuinfo |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 61 model name : Virtual CPU a7769a6388d5 stepping : 2 microcode : 0x1 cpu MHz : 2394.454 cache size : 16384 KB physical id : 0 siblings : 1 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single kaiser fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass bogomips : 4788.90 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: |
Memory Information
You can assign 512MB, 1GB, 2GB or more memory to a server on Vultr, Read my guide here on upgrading resources for Vultr VM’s here.
Only upgrade your server’s memory when server processes demand it, there is no need to pay for extra idle memory. Read my older guides on upgrading Digital Ocean and AWS servers.
I use the htop utility to monitor memory and processes. The memory usage will depend on how you have configured your server to use connection pools in code, MySQL or services. Also what memory demands do you get in pean bandwidth times?
You can check your server memory details on Ubuntu with this command
1 | cat /proc/meminfo |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | MemTotal: 2048104 kB MemFree: 96176 kB MemAvailable: 693072 kB Buffers: 183476 kB Cached: 526124 kB SwapCached: 0 kB Active: 1467220 kB Inactive: 243228 kB Active(anon): 1070464 kB Inactive(anon): 27004 kB Active(file): 396756 kB Inactive(file): 216224 kB Unevictable: 3652 kB Mlocked: 3652 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 64 kB Writeback: 0 kB AnonPages: 1004504 kB Mapped: 114664 kB Shmem: 94192 kB Slab: 192692 kB SReclaimable: 171892 kB SUnreclaim: 20800 kB KernelStack: 3072 kB PageTables: 20528 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 1024052 kB Committed_AS: 2424332 kB VmallocTotal: 34359738367 kB VmallocUsed: 0 kB VmallocChunk: 0 kB HardwareCorrupted: 0 kB AnonHugePages: 247808 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 67440 kB DirectMap2M: 2029568 kB |
Use Memory or Disk (Swap)
You can configure the use of Memory over Disk by configuring your/etc/sysctl.conf file (setting value “vm.swappiness”)
You can check your swap file settings by running the following command
1 2 | cat /proc/sys/vm/swappiness 1 |
Or By running
1 2 | sysctl vm.swappiness vm.swappiness = 1 |
Set a new swap file value by editing /etc/sysctl.conf
1 | sudo nano /etc/sysctl.conf |
Set the following to use more ram over the swap disk.
1 | vm.swappiness = 1 |
Read about swappiness values here: https://en.wikipedia.org/wiki/Swappiness
Service Performance
Performance (and allocated resources) depends on the demands of your operating system and installed software
What operating system do you have?
1 2 3 4 5 6 | lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.4 LTS Release: 16.04 Codename: xenial |
View NGINX Status, how much memory does it use?
1 2 3 4 5 6 7 8 9 10 11 12 13 | /etc/init.d/nginx status ● nginx.service - A high performance web server and a reverse proxy server Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2018-05-25 21:28:25 AEST; 1 weeks 3 days ago Docs: man:nginx(8) Main PID: #### (nginx) Tasks: 3 Memory: 58.9M CPU: 33min 11.515s CGroup: /system.slice/nginx.service ├─#### nginx: master process /usr/sbin/nginx -g daemon on; master_process on; ├─#### nginx: worker process └─#### nginx: cache manager process |
PHP (and Child Worker) status how much memory does it use and how many child workers do you have? Read my add PHP child workers post here (and update to PHP 7.2 here)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | sudo service php7.2-fpm status ● php7.2-fpm.service - The PHP 7.2 FastCGI Process Manager Loaded: loaded (/lib/systemd/system/php7.2-fpm.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2018-05-25 21:28:26 AEST; 1 weeks 3 days ago Docs: man:php-fpm7.2(8) Main PID: #### (php-fpm7.2) Status: "Processes active: 0, idle: 20, Requests: 75911, slow: 0, Traffic: 0.1req/sec" Tasks: 21 Memory: 694.2M CPU: 20h 49min 45.132s CGroup: /system.slice/php7.2-fpm.service ├─ #### php-fpm: master process (/etc/php/7.2/fpm/php-fpm.conf) ├─ #### php-fpm: pool www-acc ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-acc ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr └─ #### php-fpm: pool www-usr |
MySQL Status
1 2 3 4 5 6 7 8 9 10 | sudo service mysql status ● mysql.service - MySQL Community Server Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2018-05-25 21:28:27 AEST; 1 weeks 3 days ago Main PID: ##### (mysqld) Tasks: 35 Memory: 405.9M CPU: 2h 17min 31.822s CGroup: /system.slice/mysql.service └─#### /usr/sbin/mysqld |
Shared VM Hosts
One of the biggest impacts (after server latency) for your server is not the disk performance but the number of hosts/websites on the server who are also using the disk and server resources.
I have 80 other web servers on my server (based on a reverse lookup).
I may move to a dedicated box when I can afford it.
Security
Above all else ensure that security is number 1 priority and make performance second priority.
Scan your site with Zap, Qualys and Kali Linux. Performance means nothing if you are hacked.
Simulated concurrent users
You can use Siege to test the maximum concurrent users accessing your site before the server starts to drop connections.
FYI: If you use Cloudflare (you should) this may not work as it will block connections.
Install Siege
1 | sudo apt-get install siege |
Test your server with 10 concurrent serves for 1 minute
1 | siege -t1m c10 'https://yourserver.com/' |
Results
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | siege -t1m c10 'https://yourserver.com/' ** SIEGE 3.0.8 ** Preparing 15 concurrent users for battle. The server is now under siege... Lifting the server siege... done. Transactions: 417 hits Availability: 100.00 % Elapsed time: 59.01 secs Data transferred: 8.24 MB Response time: 1.62 secs Transaction rate: 7.07 trans/sec Throughput: 0.14 MB/sec Concurrency: 11.46 Successful transactions: 417 Failed transactions: 0 Longest transaction: 2.26 Shortest transaction: 1.49 |
Keep upping the connections (from 10 above) to a limit where connections start dropping.
I tried 25 then 50 concurrent users hitting a server on Digital Ocean and it did not fail.
Conclusion
- Choose a server near your customers
- Change hosts if one is faster and cheaper
- Measure or benchmark your server (and compare over time).
- Use Cloudflare
Create your own server today
- Create your own server on Vultr here.
- Create your own server on Digital Ocean here.
- Create your own server on UpCloud here.
And remember you can install the Runcloud server management dashboard here.
I hope this guide helps someone.
Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 3 of 4
How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 3 of 4
Advertisement:
Read Part 1, Part 2, Part 3 or Part 4
I used these commands to generate bonnie++ reports from the data in part 2
1 2 3 4 5 6 7 | echo "<h1>Bonnie Results</h1>" > /www-data/bonnie.html echo "<h2>Vultr (Sydney)</h2>" >> /www-data/bonnie.html echo "1.97,1.97,servername,1,1528177870,4G,,656,99,308954,68,113706,33,1200,92,188671,30,10237,251,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,26067us,119ms,179ms,29139us,26069us,16118us,1463us,703us,880us,263us,119us,593us" | bon_csv2html >> /www-data/bonnie.html echo "<h2>Digital Ocean (London)</h2>" >> /www-data/bonnie.html echo "1.97,1.97,servername,1,1528186398,4G,,699,99,778636,74,610414,60,1556,99,1405337,59,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,17678us,10099us,17014us,7027us,3067us,2366us,1243us,376us,611us,108us,59us,181us" | bon_csv2html >> /www-data/bonnie.html echo "<h2>UpCloud (Singapore)</h2>" >> /www-data/bonnie.html echo "1.97,1.97,servername,1,1528226703,4G,,1014,99,407179,24,366622,32,2137,99,451886,17,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,11297us,54232us,16443us,4949us,44883us,1595us,264us,340us,561us,138us,66us,327us" | bon_csv2html >> /www-data/bonnie.html |
Image of results here
Network Performace
IMHO Network Latency is the biggest impact on server performance, Read my old post on scalability on a budget here. I am in Australia an having a server in Singapore was too far away and latency was terrible.
Here is a non-scientific example of pinging a Vultr, Digital Ocean and UpCloud server in three different locations (and Google).
Test Ping Results
- Vultr 132ms Ping Average (Sydney)
- Digital Ocean 322ms Ping Average (London)
- UpCloud 180ms Ping Average (Singapore)
Latency matters, run a https://www.webpagetest.org/ scan over your site to see why.
Adding https added almost 0.7 seconds to https communications in the past on Digital Ocean (a few thousand kilometres away). The longer the latency the longer HTTPS handshakes take.
Deploying a server to Singapore (in my experience) is bad if your visitors are in Australia. But deploying to other regions may be lower in cost though. It’s a trade off.
Deploy servers as close as you can to your customers is the best tip for performance.
Also, consider setting up Image Optimization and Image CDN plugins (guide here) in WordPress and using Cloudflare (guide here)
Benchmarking with SysBench
Install CPU Benchmark
1 | sudo apt-get install sysbench |
CPU Benchmark (Vultr/Sydney)
Result
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Doing CPU performance benchmark Threads started! Done. Maximum prime number checked in CPU test: 20000 Test execution summary: total time: 39.1700s total number of events: 10000 total time taken by event execution: 39.1586 per-request statistics: min: 2.90ms avg: 3.92ms max: 20.44ms approx. 95 percentile: 7.43ms Threads fairness: events (avg/stddev): 10000.0000/0.00 execution time (avg/stddev): 39.1586/0.00 |
39.15 seconds
CPU Benchmark (Digital Ocean/London)
1 | sysbench --test=cpu --cpu-max-prime=20000 run |
Result
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Doing CPU performance benchmark Threads started! Done. Maximum prime number checked in CPU test: 20000 Test execution summary: total time: 33.4382s total number of events: 10000 total time taken by event execution: 33.4352 per-request statistics: min: 3.24ms avg: 3.34ms max: 6.45ms approx. 95 percentile: 3.45ms Threads fairness: events (avg/stddev): 10000.0000/0.00 execution time (avg/stddev): 33.4352/0.00 |
33.43 sec
CPU Benchmark (UpCloud/Singapore)
1 | sysbench --test=cpu --cpu-max-prime=20000 run |
Result
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Doing CPU performance benchmark Threads started! Done. Maximum prime number checked in CPU test: 20000 Test execution summary: total time: 23.7809s total number of events: 10000 total time taken by event execution: 23.7780 per-request statistics: min: 2.35ms avg: 2.38ms max: 6.92ms approx. 95 percentile: 2.46ms Threads fairness: events (avg/stddev): 10000.0000/0.00 execution time (avg/stddev): 23.7780/0.00 |
23.77 sec
Surprisingly, 1st place in prime generation goes to UpCloud, then Digital Ocean then Vultr. UpCloud has some good processors.
Processors:
- UpCLoud (Singapore): Intel(R) Xeon(R) CPU E5-2687W v4 @ 3.00GHz
- Digital Ocean (London): Intel(R) Xeon(R) CPU E5-2630L v2 @ 2.40GHz
- Vultr (Sydney): Virtual CPU a7769a6388d5 (Masked/Hidden CPU @ 2.40GHz)
(Lower is better)
(oops, typo in the chart should say Vultr)
Benchmark the file IO
Confirm free space
1 | df -h / |
Install Sysbench
1 | sudo apt-get install sysbench |
I had 10GB free on all servers (Vultr, Digitial Ocean and UpCloud) so I created a 10GB test file.
1 2 3 4 5 | sysbench --test=fileio --file-total-size=10G prepare sysbench 0.4.12: multi-threaded system evaluation benchmark 128 files, 81920Kb each, 10240Mb total Creating files for the test... |
Now I can run the benchmark and use the pre-created text file.
1 | sysbench --test=fileio --file-total-size=10G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run |
SysBench description from the Ubuntu manpage.
“SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load. The idea of this benchmark suite is to quickly get an impression about system performance without setting up complex database benchmarks or even without installing a database at all.”
SysBench Results (Vultr/Sydney)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Initializing random number generator from timer. Extra file open flags: 0 128 files, 80Mb each 10Gb total file size Block size 16Kb Number of random requests for random IO: 0 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Time limit exceeded, exiting... Done. Operations performed: 385920 Read, 257280 Write, 823266 Other = 1466466 Total Read 5.8887Gb Written 3.9258Gb Total transferred 9.8145Gb (33.5Mb/sec) 2143.98 Requests/sec executed Test execution summary: total time: 300.0026s total number of events: 643200 total time taken by event execution: 182.4249 per-request statistics: min: 0.01ms avg: 0.28ms max: 18.12ms approx. 95 percentile: 0.55ms Threads fairness: events (avg/stddev): 643200.0000/0.00 execution time (avg/stddev): 182.4249/0.00 |
SysBench Results (Digital Ocean/London)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Initializing random number generator from timer. Extra file open flags: 0 128 files, 80Mb each 10Gb total file size Block size 16Kb Number of random requests for random IO: 0 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Time limit exceeded, exiting... Done. Operations performed: 944280 Read, 629520 Write, 2014432 Other = 3588232 Total Read 14.409Gb Written 9.6057Gb Total transferred 24.014Gb (81.968Mb/sec) 5245.96 Requests/sec executed Test execution summary: total time: 300.0024s total number of events: 1573800 total time taken by event execution: 160.5558 per-request statistics: min: 0.00ms avg: 0.10ms max: 18.62ms approx. 95 percentile: 0.34ms Threads fairness: events (avg/stddev): 1573800.0000/0.00 execution time (avg/stddev): 160.5558/0.00 |
SysBench Results (UpCloud/Singapore)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Initializing random number generator from timer. Extra file open flags: 0 128 files, 80Mb each 10Gb total file size Block size 16Kb Number of random requests for random IO: 0 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Time limit exceeded, exiting... Done. Operations performed: 994320 Read, 662880 Write, 2121090 Other = 3778290 Total Read 15.172Gb Written 10.115Gb Total transferred 25.287Gb (86.312Mb/sec) 5523.97 Requests/sec executed Test execution summary: total time: 300.0016s total number of events: 1657200 total time taken by event execution: 107.4434 per-request statistics: min: 0.00ms avg: 0.06ms max: 15.43ms approx. 95 percentile: 0.13ms Threads fairness: events (avg/stddev): 1657200.0000/0.00 execution time (avg/stddev): 107.4434/0.00 |
Comparison
sysbench fileio results (text)
Read
- Vultr (Sydney): 385,920
- Digital Ocean (London): 944,280
- UpCloud (Singapore): 944,320
Write
- Vultr (Sydney): 823,266
- Digital Ocean (London): 629,520
- UpCloud (Singapore): 662,880
Other
- Vultr (Sydney): 1,466,466
- Digital Ocean (London): 3,588,232
- UpCloud (Singapore): 2,121,090
Total Read Gb
- Vultr (Sydney): 5.8887 Gb
- Digital Ocean (London): 14.409 Gb
- UpCloud (Singapore): 15.172 Gb
Total Written Gb
- Vultr (Sydney): 3.9258 Gb
- Digital Ocean (London): 9.6057 Gb
- UpCloud (Singapore): 10.115 Gb
Total Transferred Gb
- Vultr (Sydney): 9.8145 Gb
- Digital Ocean (London): 24.014 Gb
- UpCloud (Singapore): 25.287 Gb
Now I can remove test file io benchmark file
1 2 3 4 | sysbench --test=fileio --file-total-size=2=10G cleanup sysbench 0.4.12: multi-threaded system evaluation benchmark Removing test files... |
Confirm the test file has been deleted
1 2 3 | df -h / Filesystem Size Used Avail Use% Mounted on /dev/vda1 40G 16G 23G 41% / |
Bonus: Benchmark MySQL (on my main server (Vultr) not on Digital Ocean and UpCLoud)
I tried to run a command
1 2 3 4 5 6 | sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=#################################### prepare sysbench 0.4.12: multi-threaded system evaluation benchmark FATAL: unable to connect to MySQL server, aborting... FATAL: error 1049: Unknown database 'test' FATAL: failed to connect to database server! |
To fix the error I created a test table with Adminer (guide here).
Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4
How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4
Advertisement:
Read Part 1, Part 2, Part 3 or Part 4
Measure Disk Performance with Bonnie++
Installing Bonnie++ on Ubuntu
1 | apt-get install bonnie++ |
Read this. post on using Bonnie++
Starting Bonnie++
1 | bonnie++ -d /tmp -r 2048 -u username |
Bonnie++ Readme.
Disk io with bonnie++ on Vultr/Sydney
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.97 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP servername 4G 656 99 308954 68 113706 33 1200 92 188671 30 10237 251 Latency 26067us 119ms 179ms 29139us 26069us 16118us Version 1.97 ------Sequential Create------ --------Random Create-------- servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ Latency 1463us 703us 880us 263us 119us 593us 1.97,1.97,servername,1,1528177870,4G,,656,99,308954,68,113706,33,1200,92,188671,30,10237,251,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,26067us,119ms,179ms,29139us,26069us,16118us,1463us,703us,880us,263us,119us,593us |
Disk io with bonnie++ on Digital Ocean/London
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.97 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP servername 4G 699 99 778636 74 610414 60 1556 99 1405337 59 +++++ +++ Latency 17678us 10099us 17014us 7027us 3067us 2366us Version 1.97 ------Sequential Create------ --------Random Create-------- servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ Latency 1243us 376us 611us 108us 59us 181us 1.97,1.97,servername,1,1528186398,4G,,699,99,778636,74,610414,60,1556,99,1405337,59,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,17678us,10099us,17014us,7027us,3067us,2366us,1243us,376us,611us,108us,59us,181us |
Disk io with bonnie++ on UpCloud/Singapore
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | Writing a byte at a time...done Writing intelligently...done Rewriting...done Reading a byte at a time...done Reading intelligently...done start 'em...done...done...done...done...done... Create files in sequential order...done. Stat files in sequential order...done. Delete files in sequential order...done. Create files in random order...done. Stat files in random order...done. Delete files in random order...done. Version 1.97 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP servername 4G 1014 99 407179 24 366622 32 2137 99 451886 17 +++++ +++ Latency 11297us 54232us 16443us 4949us 44883us 1595us Version 1.97 ------Sequential Create------ --------Random Create-------- servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ Latency 264us 340us 561us 138us 66us 327us 1.97,1.97,servername,1,1528226703,4G,,1014,99,407179,24,366622,32,2137,99,451886,17,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,11297us,54232us,16443us,4949us,44883us,1595us,264us,340us,561us,138us,66us,327us |
Now read this site on how to make sense of this data
How I upgraded from a standalone 1Password 6.x licence to 1Password (7+) cloud subscription on OSX
This is how I upgraded from my standalone 1Password 6.x family licence to a 1Password 7 cloud subscription on OSX. I am not reviewing 1Password here.
Advertisement:
This is NOT a paid endorsement, this is output from my legitimate quest from upgrading an old stand-alone family licence to a cloud subscription. I have been using 1Password for the past 5 years and have recommended it to everyone I know.
Always backup your data before updating (things can go wrong), good luck. At the time of writing 1Password 7 was not out of beta.
Why
I have a number of guides on moving away from CPanel, Setting up VM’s on UpCloud, AWS, Vultr or Digital Ocean and let’s say 1Password has helped me store everything from service invoices, SSH password service passwords etc. I did have a stroke last year (caused by the flu (a cough) and luckily all is OK ) and I now realise that having everything out of my brain and in a secure vault is a good idea (touch wood).
Reasons why I use 1Password – Password Manager.
- It is a good idea to use a password manager.
- It has allowed me to have a different password on every site I use.
- I managed to stop using my old “same” password on every site I used since the 1990’s.
- I am paranoid (check out https://howsecureismypassword.net/, https://haveibeenpwned.com/ and google “sites hacked” or “passwords exposed”).
- It’s a good idea.
1Password 7 News
1Password 7 has been hitting my twitter timeline, should I upgrade? Here is the official upgrade guide.
Security Researcher Troy Hunt’s – https://haveibeenpwned.com/ is now a feature in 1Password 7
@1Password just keeps getting better and better. Ping: @troyhunt pic.twitter.com/qTtE6XyoXb
— Grant Harrington (@harringg) May 22, 2018
I wrote a PHP implementation to check a password exposure level with Troy Hunt’s pwned passwords API and know it’s a good idea.
Also, there are loads of great features in 1Password 7.
Anything that can help create secure passwords is a good idea.
86% of Passwords are Terrible (and Other Statistics) https://t.co/pSqbb7IV0g by @troyhunt
— Particular Software (@ParticularSW) May 25, 2018
1Password Twitter Support Shoutout
Before I begin I would like to acknowledge the patient 1Password support team on twitter. They answered well over 20 questions from me and handled my frustrations of there not being a clear standalone family licence, I suspected a plot to force people onto a cloud subscription at first.
In an ideal world upgrading, 1Password should be an easy process (1Password Twitter Support indicated)
Load’s of 1 Password activity on Twitter
Phew! ? 1Password 7 for Mac has generated a lot of excitement! If you have any questions, be sure to check out our forums to see if they’ve been answered. Our team is always here. https://t.co/Xixe8e80yY
— 1Password (@1Password) May 23, 2018
Before I downloaded the latest 1Password 7 I fired heaps of questions at the twitter support. I hope 1Password give them a raise or bonus.
I did spend way too long reading past the negative 1Password support posts on “where is the standalone licence”, “beta discounts are gone”, “why so expensive” and “how can I upgrade from 1password 6 and still use dropbox” etc.
I ended up logging a support ticket (looking for the unicorn beta tester discount/stand-alone licence, I think I was too late to join the beta program).
I backed up my 1Password 6 data
Always take backups of your data before upgrading anything.
- Open your existing 1Password 6
- Click File, Export, All Items
- Choose a File Name and press Save
- Verify Data (open Finder)
I also backed up the 1Password file in Dropbox before upgrading. Simply drag it to your desktop.
Advertisement:
I visited https://1password.com/extlink/signin/ and…
- Signed in (create an account if you don’t have one), I had one from a previous support ticket I logged.
- I chose “Individual” account type.
- Specified my Email Address
- Verified My Email
Then I..
- Added my Name
- Skipped the Credit card option (to activate the trial)
- Generated a master key (this is combined with your password)
- Created a new password for a new password vault
- Logged in
- Saved my Emergency Recover PWF (with details)
I did have a peek at the 1Password SSL certificate strength and other tools and they came up all good (I don’t want to use an insecure service).
You too can test SSL on sites with https://dev.ssllabs.com/ssltest/
The only concern I have is TLS 1.3 is not an option yet. I use it on my blog’s web server (guide here) also a few SSL labs identified weak cyphers are presented as available from the server (Is this an issue)?
I also had a look at Google Chrome’s developer console to see if anything out of the ordinary was popping up? The console appears a little chatty? TLS 1.2 was in force securing the client/server communications so that’s nice.
Now that I am logged into my cloud 1Password (trial) account I can…
- Add/View/Edit/Delete items in my vault.
- Download desktop/mobile apps.
- Import data to vaults.
- Turn on Two Factor authentication.
- Create/Edit/Delete a password vault (or set as default).
- Update billing details (if you wish to subscribe)
- Contact Support
Now I can connect my new 1Password cloud account to my local 1Password 6 installation by.
- Open 1 Password (on my Mac)
- Open the Preferences and go to the Account tab
- Click on Scan the account details
- Move the scan window over a QR Code (Setup Code) in a logged in 1password.com screen (login detail like login server, master key and username are auto-entered).
- Enter my vault password.
Advertisement:
Now I am prompted to import my local 1Password data into the cloud account from my local 1Password.
When the import completed I was prompted to delete the local vault (I said yes because I backed it up).
Tip: 1Password 6 on my Mac did not appear to delete the Dropbox data so I deleted this manually.
After a few minutes, I noticed Dropbox was still syncing files?
Troubleshooting: I had to set my new cloud vault as the primary vault to save to and not the old vault that was syncing via Dropbox. I also deleted all links to Dropbox on iOS and Android devices.
I did notice that 1Password was 6.8.9 (I thought 1password 7 was the latest?, I did try the update button). I ended up ticking “Include beta builds” and then 1Password 7.0 is a download option (maybe this will change in the next few days)?
I opened 1Password 7 on my local desktop.
I had a quick look around in 1Password 7 for the https://haveibeenpwned.com/ feature. I opened an existing account I added to 1Password. It look’s nice.
Some nice alerts and features I noticed when viewing my data in 1 Password 7.
Aside: I had to opt into beta builds on Windows to get 1Password 7 too.
Summary
When I set out and wanted a stand-alone licence but it appears I would need to pay for a licence on Windows and Mac and portable devices.
I overlooked an earlier DM from 1Password (that provided the purchase links) so I decided to go with a subscription (I think I missed the BETA program too, no reply from the hockey app email when opting into beta on Windows).
Buy standalone licences
- Buy Mac Licence ($39 in BETA or $64.99 RRP)
- Buy Windows Licence ($39 in BETA or $64.99 RRP)
From what I could see standalone licences only work via Dropbox (or locally) and not via the 1Password cloud.
However, the subscription does away with the requirement to buy multiple licences (all apps are free once you subscribe). I am not sure when 1Password 8 is coming out so I think it is wiser to go with a yearly subscription (that’s about 10.8c a day in Australian peso’s).
Time to Subscribe
I pulled the trigger and subscribed 🙂
Advertisement:
One nice thing is the trial time is added on to the subscription length so if you have 30 days left in the trial it add’s on to the yearly subscription length (13 months), that’s nice.
Conclusion
Happy = Yes (they are shooting fish in a barrel)
Could have been easier to upgrade from 1 password = Yes
I hope this guide helps someone.
Find out more about 1Password at http://1password.com/
Ask a question or recommend an article
Revision History
v1.3 Fixed typo in the title/url.
v1.2 Added Links
v1.1 Added Conclusion
v1.0 Initial post
How to backup WordPress on a host that has CPanel
This is a simple guide that demonstrates how you can log in to a host that offers the CPanel tools to backup all of your website files (and databases). Backing up your website should be done often and especially before you migrate to any another website host. I used to change hosts every few years (they don’t own your site, you do).
Advertisement:
I have a number of guides on moving away from CPanel, setting up VM’s on UpCloud, AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line copying files to a server via command line editing remote files locally etc but how do you manage a website with CPanel?
You can normal login to CPanel tools on a shared host by loading www.yourdomainnam.com/cpanel (failing that login to your domain hosts web GUI and find your CPanel interface there).
Step 1: Login to your Host
Login to your web host
Step 2: Find your CPanel Interface
Hosts are a bit different but in this case, I just click my domain to find the CPanel link.
I found it, I clicked the CPanel login.
Step 3: CPanel Applications
CPanel does offer good tools to manage your websites like web-based File Manager and Database tool called phpMyAdmin.
Aside: CPanel/Hosts Downsides
The thing I don’t like about hosts that offer CPanel is they usually limit delivery of your website to extract more money. Nothing worse than receiving Resource Limit Is Reached errors.
Also shared hosts usually lag way behind in newer software versions like PHP and MySQL (this is a security concern).
TIP: You can scan your site for vulnerabilities using Qualsys Freescan, Zap or Kali Linux.
Here is a security scan of a shared host (with CPanel) that I was using in 1999. Note the high vulnerabilities and old version of Linux.
Also, a shared host will often overcharge you (e.g $150 a year) for a poorly configured SSL certificate.
This was an SSL cert I paid $150 a year for (evaluated with SSL Labs SSL Test) on a shared host with CPanel.
Aside: Self Managed Upsides
After I moved my domain to a self-managed virtual machine I migrated WordPress, set up a free SSL certificate, sped up my site with a CDN, setup Cloudflare, setup better TLS security etc
When you manage your own server you can install a free SSL certificate in under 1 minute.
Below is my SSL certificate. A strong SSL certificate will increase search engine traffic
Aside: Compare Shared host speed v Self Managed
FYI: https://gtmetrix.com/ is a great site for measuring the speed of a website (shared of self-managed). I found great speed improvements after moving away from a host offering CPanel, tweaking the server and setting up cloudflare. A self-managed server will allow you to tweak anything you want.
GTMetrix results:
I like how self-managed servers allow you to scale the server’s resources yourself, move servers or add storage etc.
Aside: SSL Certificate
If you have an SSL cert you should test it often as vulnerabilities pop up from time to time.
FYI: All sites will soon require an SSL certificate to be sent traffic from search engines (no SSL = lower traffic).
SSL Test my site: https://dev.ssllabs.com/ssltest/analyze.html?d=fearby.com&s=104.27.154.69
Now enough with the self-managed serve asides and back to how to backup your website with CPanel tools.
Step 4: Backup your web files in CPanel
Use the File Explorer app in CPanel
Highlight all files that you want to backup (highlight everything but not past backup files).
View the files to compress summary
Click Compress Files(s) and view the backup progress
You can now download the backup zip file in your browser (click the file and click Download).
Download Progress.
Step 5: Backup your database in CPanel
Now we need to backup any MySQL database(s) that may be used by WordPress
Open the phpMyAdmin app in CPanel.
FYI: Alternatively, you can use a free tool called Adminer to backup and restore our database.
Click your WordPress database (on the left). You can identify your current WordPress database by opening the wp-config.php file.
The first step is to perform an online cold backup of the WordPress database.
- Click the database you want to backup (on the left)
- Click the Options tab
- Under the Copy To frame enter a new database name to copy the database too
- Click Go
Now you have an online cold spare that you can use just in case the original database corrupts itself. You can rename the database or configure WordPress to point to this new database if need be.
Now let’s download a copy of the database (Repeat for multiple databases).
- Click the database you want to backup
- Click the Export tab
- Click Go
You should now have a backup of your website in a zip file and an export of your database in a .sql text file, SQL files can be re-imported to databases later.
TIP: Backup often.
I hope this guide helps someone.
Ask a question or recommend an article
Revision History
v1.0 Initial post
How to install Windows 10 Pro alongside an OSX partition with Apple Boot Camp
This guide will show you how to install Windows 10 Pro alongside an OSX partition with Apple Boot Camp.
Advertisement:
I have a number of guides on moving away from CPanel, Setting up VM’s on UpCloud, AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. I use OSX to develop and occasionally need to use Windows to achieve some development tasks, this is how you can install Windows on an Apple PC.
Apple Boot Camp
Apple Bootcamp software can be found here: https://support.apple.com/en-au/HT201468
Bootcamp Steps.
You can download a Windows 10 ISO here: https://www.microsoft.com/en-au/software-download/windows10
You will need to plug in a 16GB or larger USB key to use as an install medium.
Download Boot Camp
Download AppBoot Camp here: https://support.apple.com/en-au/boot-camp
Start Apple Boot Camp, click Continue, Tick all options, select the Windows 10 ISO file (beware Bootcamp will select the first ISO file it finds, ensure your USB is selected, Click Continue (tip: You may need to er format and prepare the USB key 10 times, Apple Bootcamp is not the most reliable program), It may take 6 hours to copy files pre Window Setup).
Choose your Windows partition size (at least 200GB is ideas)
When the Bootcamp Wizard is complete you can reboot into Windows (automatic at the end of the wizard)
Enter the desired details in each step (e.g Local, Windows Key, Windows Version, Partition and Country etc)
Loads more steps like Keyboard Layout, Domain, Microsoft Account, Password and Cortana and Privacy etc.
Windows is now installed, Apple Bootcamp will set up appropriate drivers for your Mac,
While this is happening I will install Google Chrome Canary.
Bootcamp has now finished setting up drivers.
Windows 10 Start menu (I prefer OSX’s simplicity)
You can set up your prefered Startup disk in the Apple System Preferences or press the Option key on startup and choose a partition to boot.
Update: Open a Windows 10 Boot Camp Installation on OSX in Parallels (like a VM)
Read this guide will show you how you can open a Windows 10 Boot Camp Installation on OSX in Parallels (like a VM).
I hope this guide helps someone.
Ask a question or recommend an article
Revision History
v1.1 Added Open Bootcams as VM
v1.0 Initial post