ubuntu

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4

June 5, 2018 by Simon

How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4

Read Part 1, Part 2, Part 3 or Part 4

Measure Disk Performance with Bonnie++

Installing Bonnie++ on Ubuntu

apt-get install bonnie++

Read this. post on using Bonnie++

Benchmark disk IO with DD and Bonnie++

Starting Bonnie++

bonnie++ -d /tmp -r 2048 -u username

Bonnie++ Readme.

Disk io with bonnie++ on Vultr/Sydney

Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
servername 4G 656 99 308954 68 113706 33 1200 92 188671 30 10237 251
Latency 26067us 119ms 179ms 29139us 26069us 16118us
Version 1.97 ------Sequential Create------ --------Random Create--------
servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 1463us 703us 880us 263us 119us 593us
1.97,1.97,servername,1,1528177870,4G,,656,99,308954,68,113706,33,1200,92,188671,30,10237,251,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,26067us,119ms,179ms,29139us,26069us,16118us,1463us,703us,880us,263us,119us,593us

Disk io with bonnie++ on Digital Ocean/London

Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
servername 4G 699 99 778636 74 610414 60 1556 99 1405337 59 +++++ +++
Latency 17678us 10099us 17014us 7027us 3067us 2366us
Version 1.97 ------Sequential Create------ --------Random Create--------
servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 1243us 376us 611us 108us 59us 181us
1.97,1.97,servername,1,1528186398,4G,,699,99,778636,74,610414,60,1556,99,1405337,59,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,17678us,10099us,17014us,7027us,3067us,2366us,1243us,376us,611us,108us,59us,181us

Disk io with bonnie++ on UpCloud/Singapore

Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
servername 4G 1014 99 407179 24 366622 32 2137 99 451886 17 +++++ +++
Latency 11297us 54232us 16443us 4949us 44883us 1595us
Version 1.97 ------Sequential Create------ --------Random Create--------
servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 264us 340us 561us 138us 66us 327us
1.97,1.97,servername,1,1528226703,4G,,1014,99,407179,24,366622,32,2137,99,451886,17,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,11297us,54232us,16443us,4949us,44883us,1595us,264us,340us,561us,138us,66us,327us

Now read this site on how to make sense of this data

< Previous – Next >

Read Part 1, Part 2, Part 3 or Part 4

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4

June 2, 2018 by Simon

How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4. Update: I moved my domain to UpCloud.

Update (June 2018): I moved my domain to UpCloud (they are that awesome). Use this link to signup and get $25 free credit. Read the steps I took to move my domain to UpCloud here.

Comparing Digital Ocean/Vultr and UpCloud Disk IO

I have a number of guides on moving away from CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean (all in the search of extra performance) but how do you know when a server performance is ok apart from running GT Metrix and other external site benchmarking tools.

This post is split up as it was too long.

Read Part 1, Part 2, Part 3 or Part 4

Spoiler: It all depends on where your server is located and what you do with it (Tweaks will improve the performance).

P.S This is NOT a paid endorsement or conclusive test (just a quick benchmark/review).

What does your server do?

You need to know what your server does 24/7 and what resources the services need.

I use htop to view real-time and historical usage data for each process.

Tweaking Advice

A friend gave me good advice re-tweaking a cheap host to get good performance

yeah but you are trying to get speed out of budget hosting. Good, fast, cheap, pick 2.
— Kerry Hoath (@khoath) June 2, 2018

I am not a fan of just throwing more money at a host and expecting better performance. Host have unique features and cons., there is no shortage of hosts or host cons.

How can you run synthetic benchmarks to determine comparable performance metrics?

WARNING: Comparing synthetic benchmarks can be far removed from real-world speeds. Benchmark results below were from 3 different servers I have on 3 different hosts in three different locations (the only thing the same was the use of Ubuntu 16.04 $5/m servers). These results are not scientific and should not be used to compare host providers. Benchmark runs were one-off (not averages over multiple timezones/days).

Disk Performance

Speaking of disk performance I noticed this the other day on the RunCloud blog. Faster than SSD (UpCloud)?

UpCloud Faster-than-SSD Cloud Hosting Server (Promo Code Inside)

Runcloud is a server management console that can interface with your domains (read my old review here). I don’t use Runcloud but it is great for those who need a GUI to help manage VM via a dashboard. However, I prefer to know what is going on under the hood. I have investigated webmin in the past though.

Let’s do a quick IO benchmark test between UpCloud, Digital Ocean and Vultr on similarly low end $5/m servers,

Good advice on command line benchmarking tools from a friend.

depends on what sort of load you want to simulate. iozone is old but reliable. bonny might give you more figures you want.
— Kerry Hoath (@khoath) June 2, 2018

Installing iozone to test disk performance

I searched for a post on using iozone (Thanks thegeekstuff). I will be reviewing the “Writer report” and “Reader report”. Read more about iozone here.

View the iozone page for how to break down results.

(image snip from http://www.iozone.org/)

Install iozone on Ubuntu

sudo apt-get install iozone3

Run an iozone disk test and output the results to a spreadsheet.

iozone -a -b iozone.xls

Now let’s run a Read/Write test on Vultr/Digital Ocean and UpCloud. Multiple runs were not performed, this is not a scientific test (just a simple benchmark test (as is, ignoring sever load and local infrastructure/timezone load)).

iozone Benchmark results for Vultr “Read” (Sydney)

	“4”	“8”	“16”	“32”	“64”	“128”	“256”	“512”	“1024”	“2048”	“4096”	“8192”	“16384”
64	2133730	3363612	4274062	4564786	6421025
128	2248149	3536566	4135958	7082197	4135958	11720614
256	1884399	2699161	3879045	3667079	5971678	5687020	5687020
512	3140488	3736016	3684733	4262523	4610256	2638816	5067142	5684095
1024	1617808	1939207	3411938	3999762	4048778	4614246	3083680	5885083	6609617
2048	1926510	2569678	4423683	4997618	3937075	459605	2896324	3542524	4971585	4707314
4096	1701683	2151300	4209920	5001700	4751325	4869845	5389246	3647681	4928521	6207035	4347346
8192	2063424	2329346	3203763	2937280	3221485	3232699	3626431	3650706	3789200	4110603	3715045	4350542
16384	1738553	2778362	3397613	3679205	3693442	3171501	3524291	3393586	3004024	3552531	3456574	2693845	2488861
32768	0	0	0	0	2952894	3537153	3574875	3768155	4719613	3890280	3394995	2735222	2542914
65536	0	0	0	0	4057489	3610789	3619967	3800078	3275327	3591212	3607188	1770426	2826659
131072	0	0	0	0	3552270	1890742	5275167	3727339	3527607	1753893	3234736	2341111	1378601
262144	0	0	0	0	3798586	1302021	1491429	3712825	3228816	3757963	3715510	2592485	2481061
524288	0	0	0	0	2758756	2487923	3705741	1807328	2118309	3675988	3196367	3394330	2396842

iozone Benchmark results for Digital Ocean “Read” (London)

	4	“8”	“16”	“32”	“64”	“128”	“256”	“512”	“1024”	“2048”	“4096”	“8192”	“16384”
64	4564786	7100397	9006179	10402178	12902017
128	4717434	7082197	8548124	9795896	10567140	10779307
256	4840911	7073132	8271916	9868433	10148242	10651598	1E+07
512	4742616	6909408	8140399	9304292	9638369	10044089	1E+07	10044089
1024	4249053	5917516	6208343	7537599	9300377	10454984	7E+06	7113161	9946527
2048	3885431	6967792	6603549	6845629	10401883	9808036	9E+06	7903836	9308497	7817519
4096	2506983	5953231	6263611	6953144	7774379	6225028	6E+06	8081580	7683972	8081580	8240513
8192	3665114	4850463	5479317	6141364	6277120	6108608	6E+06	6569983	5732541	7166033	6633402	5479317
16384	3673501	4828584	5416182	6187150	6614761	6298872	6E+06	6430310	5984033	6402750	6046159	4791883	3405527
32768	0	0	0	0	4692542	6140929	6E+06	6295642	5231224	6545707	5781108	4513475	3702577
65536	0	0	0	0	6315430	5830131	6E+06	6444695	6219125	6473838	5338595	4248118	3679324
131072	0	0	0	0	6130002	6461496	6E+06	5958068	5983423	6387547	6138078	3994888	3602079
262144	0	0	0	0	6456746	6323727	6E+06	6504146	6390176	6486151	6433963	3955165	3654188
524288	0	0	0	0	1667337	6381456	6E+06	6445708	6448714	6421071	5981200	4155185	3770740

iozone Benchmark results for UpCloud “Read” (Singapore)

	“4”	“8”	“16”	“32”	“64”	“128”	“256”	“512”	“1024”	“2048”	“4096”	“8192”	“16384”
64	6421025	6421025	10821524	12902017	15972885
128	4889281	6406138	9129573	10779307	14200794	14200794
256	5320671	3879045	10758322	8815202	10245071	12812277	12228612
512	4305250	5115422	8844453	8234036	7091952	8394979	7540170	10235583
1024	4339202	4762630	5821271	6163794	6819511	4674510	6479979	8183918	10230845
2048	4204968	5319484	5800851	5816563	6243566	6378005	5953632	6851089	7940367	8229438
4096	4526013	5556581	4817948	5404504	7301864	5759634	5810280	6007355	6919538	8620945	6281934
8192	4298295	5019093	5927357	6036702	6781341	6082655	5855636	6527546	6553692	6792065	6466126	4437634
16384	4282172	5849558	6313919	6635840	6741958	6657054	6423097	5536622	6558575	6442970	4527032	3784777	3901898
32768	0	0	0	0	5825460	5423408	6504198	6665385	6365329	6426343	5263076	3718605	3705971
65536	0	0	0	0	6908075	6623116	6493259	6609738	6311805	6483610	5489674	4035982	3561526
131072	0	0	0	0	5650180	5718949	2465429	5391253	3495911	5784844	5367408	3733490	3582175
262144	0	0	0	0	6814627	6691250	6189661	5906786	6081645	5799913	5247919	4121250	3637601
524288	0	0	0	0	6404764	6309263	5673979	5751609	6288245	6305103	5978680	3911984	3767116

iozone Benchmark results for Vultr “Write” (Sydney)

	“4”	“8”	“16”	“32”	“64”	“128”	“256”	“512”	“1024”	“2048”	“4096”	“8192”	“16384”
64	289322	532815	507625	429630	566551
128	398921	465304	434078	417212	669577	821147
256	530031	613985	820398	474937	891956	815414	370025
512	387576	754083	709019	819085	702295	609421	924123	496091
1024	297233	448522	716089	923488	854073	817340	1203137	1072453	601636
2048	408697	634655	695383	1358134	549657	1295458	821154	797520	964207	258493
4096	236150	433804	1215774	1245025	820832	809958	1371339	914269	921083	1004682	1481431
8192	611113	666677	806286	715219	779825	824294	875947	870091	1046378	791192	1023592	453248
16384	435454	706149	718313	845499	893495	888068	812778	842885	820591	941120	839610	862672	406590
32768	0	0	0	0	465196	786067	938881	627294	890917	968147	872369	871329	842843
65536	0	0	0	0	515057	790172	937568	915601	897235	867197	907562	852002	743856
131072	0	0	0	0	501091	480492	813147	870886	880239	805333	684630	1117578	633185
262144	0	0	0	0	387126	323185	323656	473258	405744	369599	422554	468992	453563
524288	0	0	0	0	325588	380450	392965	451608	303255	355148	386250	432054	416512

iozone Benchmark results for Digital Ocean “Write” (London)

	“4”	“8”	“16”	“32”	“64”	“128”	“256”	“512”	“1024”	“2048”	“4096”	“8192”	“16384”
64	831569	566551	1279447	1363961	1392258
128	652488	1319723	1421023	990891	1663139	1561553
256	1185399	1152323	1534342	1598292	1826695	1707589	1514860
512	1166599	1296159	1399189	1620980	1620980	1361920	1589779	1672748
1024	1079190	1321200	1584972	1917562	1592612	1701108	1718120	1462960	1643814
2048	1210394	1470172	1621719	1550584	1796378	1643753	1713598	1759581	1649117	1488257
4096	916513	1287575	1574718	1406594	1742237	1734148	1652418	1583280	1599346	1661045	1533532
8192	1109745	1318748	1178567	1544201	1502340	1371492	1466747	1499521	1479759	1564878	1291292	1347609
16384	1106205	1282084	1374037	1503649	1429398	1461407	1496119	1578132	1547289	1333431	1203371	1198815	1501316
32768	0	0	0	0	1270914	1406589	1513114	1468226	1558303	1552038	1516336	1443280	1440360
65536	0	0	0	0	1319322	1327984	1311504	1411955	1266988	1359645	1386446	1347092	1368295
131072	0	0	0	0	1100658	1229326	1227197	1318631	1265552	1233306	1227747	1237896	1233502
262144	0	0	0	0	1167160	1064078	1155828	1185185	1086152	1193673	1080872	1062611	1141960
524288	0	0	0	0	977835	1124816	1052757	1219183	1128972	1140177	1091954	1141635	1132063

iozone Benchmark results for UpCloud “Write” (Singapore)

	“8”	“16”	“32”	“64”	“128”	“256”	“512”	“1024”	“2048”	“4096”	“8192”	“16384”
64	1143223	1255511	1562436	1452528	1279447
128	1451764	1406136	1543594	1504659	1852520	1749872
256	1642294	1829808	1970871	1855098	1802167	1952947	2000242
512	1537424	1854787	1801873	2294796	1983258	2124526	1895721	1417662
1024	1434138	1553442	1609925	1931359	2098375	2044438	1872419	1768345	1892218
2048	1562145	1901771	1817281	1848169	1967097	1296240	2267786	2081497	1915768	2007554
4096	1625372	1966378	1924741	1342092	1950306	2078175	1914873	1459656	1995152	2102849	1326855
8192	1444062	1808330	1956503	1924397	2127300	2042328	2135630	1986478	2062557	2061319	1337016	1812049
16384	1667066	1820248	1898495	2051339	2012530	2111080	2119806	1491217	2060875	1974254	1934789	1815823	1921911
32768	0	0	0	0	2057506	1454537	2075621	2070899	1869795	2052896	1892347	1855382	1873440
65536	0	0	0	0	2067127	2077673	2088994	2179809	2087471	2099108	1904723	1642505	1832204
131072	0	0	0	0	1234663	1824959	1304340	1775514	1287481	1560379	1631992	1085609	1675467
262144	0	0	0	0	685774	808487	823824	662524	681762	548308	814946	645663	732176
524288	0	0	0	0	547296	517384	503422	521173	538714	518429	528950	529593	512944

Here is my quick unscientific take on a one pass benchmark results above.

These results need some decoding.

Next >>

Read Part 1, Part 2, Part 3 or Part 4

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Update (June 2018): I moved my domain to UpCloud (they are that awesome). Use this link to signup and get $25 free credit. Read the steps I took to move my domain to UpCloud here.

Revision History

v1.2 added the fact that I Moved to UpCloud.

v1.1 Re ran iozone -a -b iozone.xls on all servers.

v1.0 Initial post

Measuring VM disk performance on Ubuntu and comparing Vultr, Digital Ocean and UpCloud

June 2, 2018 by Simon

How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud.

Read the full article here: https://fearby.com/article/measuring-vm-performance-cpu-disk-latency-concurrent-users-etc-ubuntu-comparing-vultr-digital-ocean-and-upcloud-1-4/

Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX

May 15, 2018 by Simon

This guide will aim to inform you of strong cryptographic protocols and ciphers to use on a web server on Ubuntu 16.04 and NGINX.

Read the full article here: https://fearby.com/article/setting-strong-ssl-cryptographic-protocols-and-ciphers-on-ubuntu-and-nginx/

Moving an Ubuntu 16.04 VM on Vultr from one data centre to another via snapshots

April 17, 2018 by Simon

This guide will show how you can move an Ubuntu VM server domain between Vultr data centres via snapshots.

I have a number of guides on moving away from CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Sometimes you need to move a sever between locations and/or upgrade the server (to have more memory t install WordPress).

Moving an existing Vultr server

If you don’t have an Ubuntu server click here (follow this guide).

Login to Vultr and specify a source server, click Snapshots and click Take Snapshot.

Wait for the snapshot to finish (It may take 1 hour).

Great, the snapshot is done.

Now I can create a new server (in a different data centre).

Deploy New Instance

Choose a location (Australia is at capacity, so I’ll deploy to Silicon Valley then move again in a few weeks), choose the snapshot to restore, choose a plan, I enabled IPV6/Auto Backups and Private Networking.

TIP: The password for the server will be the same as the source server so write it down.

Click Deploy Now

After a few minutes, you can see the new servers IP address, you can log in to your domain name provider (in my case Namecheap) and update the target IPV4 and IPV6 address.

You can find IPV4 and IPV6 addresses by opening your server, clicking settings then IPxV4 or IPV6.

You will need to update Vultr DNS settings (login to Vultr, Click Servers, Click DNS then edit your existing Domain DNS entry). Add you’re new serves IP addresses.

Update: I added an IPV6/AAAA record too.

Wait for DNS Replication

Goto https://www.whatsmydns.net/ and check the global DNS propagation for your new domain’s server.

If you are happy that the server has been migrated (snapshot restored) and that the domain DNS is pointing to your new server you can delete the old server in the Vultr server list.

Post-Migrate Actions

Setup Daily backups.
Review firewall settings (guide here).
Optional: Install MySQL
Optional: Install PHP
Optional: Install PHP Pooled Connections
Optional: Install WordPress
Optional: Install WordPress CDN
Optional: Configure Cloudflare
etc

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.1 Vultr Link

v1.0 Initial post

Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare

April 5, 2018 by Simon

This guide will show you how to enable the latest Transport Layer Security (TLS) 1.3 protocol with it’s predecessor Secure Sockets Layer (SSL) with NGINX and OpenSSL for better website security on an Ubuntu 16.04 server

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Making sure your server is up to date and running the latest SSL software is important. I have updated Open SSL before and blogged about this here. Do backup your server before changing settings and if you use Cloudflare (if you don’t do it now) enable Development Mode (and disable caching until changes are made).

For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

TLS 1.3 is the latest SSL security protocol that can be used between clients and servers to encrypt connections on the web.

TLS 1.3 uptake is only 60% according to https://caniuse.com/#search=TLS%201.3

Read why TLS 1.3 is important and news on TLS 1.3 can be found here: https://www.openssl.org/blog/blog/2018/02/08/tlsv1.3/

The Good and Bad

Done be like this commercial site with very poor security (tested with SSL labs and asafaweb)

Here is what the top 1 million sites do

Here it is!! Alexa Top 1 Million Analysis – February 2018 https://t.co/TjBHNX7zTi
— Scott Helme (@Scott_Helme) February 26, 2018

Installing Open SSL on Ubuntu

Connect to your Ubuntu 16.04 server via SSH (I connected to my Vultr server)

Check what version of OpenSSL you have? My OpenSSL is out of date.

# openssl version
OpenSSL 1.1.0g  2 Nov 2017

Tip: What Ciphers does your Open SSL Support?

openssl ciphers -s -v
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=RSA  Enc=CHACHA20/POLY1305(256) Mac=AEAD
DHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=DH       Au=RSA  Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA384
DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256
ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA256
DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
ECDHE-ECDSA-AES256-SHA  TLSv1 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA1
ECDHE-RSA-AES256-SHA    TLSv1 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA1
DHE-RSA-AES256-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA1
ECDHE-ECDSA-AES128-SHA  TLSv1 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA1
ECDHE-RSA-AES128-SHA    TLSv1 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA1
DHE-RSA-AES128-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA1
AES256-GCM-SHA384       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(256) Mac=AEAD
AES128-GCM-SHA256       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(128) Mac=AEAD
AES256-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA256
AES128-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA256
AES256-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA1
AES128-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA1

Time to update Open SSL

OpenSSL 1.1.1 beta is available and supports TLS 1.3 but it is n BETA form. OpenSSL code is available here.

I did the following to download and build the latest version of OpenSSL.

mkdir /openssltemp
cd /openssltemp
sudo git clone git://git.openssl.org/openssl.git
cd openssl/
./config --prefix=/usr/local/ssl --openssldir=/usr/local/ssl -Wl,-rpath,/usr/local/ssl/lib
make
sudo make install

I tried to check the open SSL version but had an error?

openssl version 
openssl: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by openssl)
openssl: /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1: version `OPENSSL_1_1_1' not found (required by openssl)

A quick GitHub ticket revealed I needed to set a path variable.

export LD_LIBRARY_PATH=/usr/local/lib
echo "export LD_LIBRARY_PATH=/usr/local/bin/openssl" >> ~/.bashrc

Open SSL now reports it’s version.

openssl version
OpenSSL 1.1.1-pre3 (beta) 20 Mar 2018

What version NGINX do you have (1.13 supports TLS 1.3) read here

# nginx -v
nginx version: nginx/1.13.9

Backup your NGINX

Do backup your server files and take a snapshot if need be. I am not responsible;e for a broken server,

sudo cp -R /etc/nginx/ /nginx-backup-26thMar-2018

Edit NGINX Configuration

Update NGINX configuration: /etc/nginx/sites-available/default

ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256;
ssl_prefer_server_ciphers on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
ssl_ecdh_curve secp384r1;

tip: Review other NGINX hardening settings here. Also remove TLSv1.0

I tested my NGINX config loaded them and restarted NGINX

nginx -t
nginx -s reload
/etc/init.d/nginx restart

Check the status of NGINX

# /etc/init.d/nginx status

[ ok ] Restarting nginx (via systemctl): nginx.service.
● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) 
     Docs: man:nginx(8)
  Process: 15154 ExecStop=/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid (code=exited, status=0/SUCCESS)
  Process: 15162 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
  Process: 15159 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
 Main PID: 15166 (nginx)
    Tasks: 4
   Memory: 2.3M
      CPU: 27ms
   CGroup: /system.slice/nginx.service
           ├─15166 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
           ├─15170 nginx: worker process
           ├─15171 nginx: cache manager process
           └─15172 nginx: cache loader process

If you have configured Cloudflare then log in and enable TLS support.

Enable TLS 1.3 in Chrome by visiting chrome://flags/#tls13-variant This should be automatic in later versions of Chrome and other browsers.

Verify TLS

I used the developer tools in Chrome to confirm the page was verified in TLS 1.3.

Updated to 1.1.1-pre6-dev

mkdir /temp
cd /temp
sudo git clone https://github.com/openssl/openssl.git
cd openssl/
./config --prefix=/usr/local --openssldir=/usr/local -Wl,-rpath,/usr/local
make
sudo make install
openssl
OpenSSL> version
OpenSSL 1.1.1-pre6-dev  xx XXX xxxx
OpenSSL> exit

Don’t forget to test your SSL strength with https://dev.ssllabs.com/ssltest/

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.4 fixed typo

v1.3 added bad ssl cert.

v1.2 ssl test v1.1 updated to 1.1.1-pre6-dev

v1.0 Initial post

Backing up OSX or an Ubuntu server with Backblaze B2 Cloud Storage from the Command Line

March 14, 2018 by Simon

This computer will show you can back up computer or server with Backblaze B2 Cloud Storage from the Command Line n OSX and Ubuntu.

This post is still being written. I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Also, I have blogged about how you can add block storage to a Vultr server, backup and restore snapshots , syncing files with rsync along with using GitHub and Bitbucket but what do you do if you need to backup large amounts of data?

Backblaze has a Cloud storage solution that costs as low as $0.005c a GB (a month), The first 10G is free. Backblaze say “From bytes to petabytes Backblaze B2 is the lowest cost high-performance cloud storage in the world. ”

Back Blaze have open sourced internal drive enclosure designs and drive failure stats and it’s time I gave them a try.

Goto https://www.backblaze.com

Create or sign in.

After you login got the dashboard.

Click Backblaze B2 Cloud Storage

Create a Bucket

Name the bucket (long names with a GUID are good).

You can rename the bucket here and change public/private and or upload/download files manually.

The first thing I did was limit the versions of files under the lifecycle settings for the bucket.

Now I created a series of subfolders to store files from different servers (I could have used many buckets but one bucket will do).

I can upload files via the Backblaze bucket GUI if I needed to.

Back Blaze has a command line tool for uploading: https://www.backblaze.com/b2/docs/quick_command_line.html

Install Steps

Backblaze state “The B2 command-line tool is available from the Python Package Index (PyPI) using the standard pip installation tool. Your first step is to make sure that you have either Python 2 (2.6 or later) or Python 3 (3.2 or later) installed.”

I have Python 2.7 installed

python --version
Python 2.7.10

Install PIP

sudo easy_install pip
Password:
Searching for pip
Best match: pip 1.5.6
Processing pip-1.5.6-py2.7.egg
pip 1.5.6 is already the active version in easy-install.pth
Installing pip script to /usr/local/bin
Installing pip2.7 script to /usr/local/bin
Installing pip2 script to /usr/local/bin

Using /Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip

I ran into issues updating b2 CLI

sudo pip install --upgrade b2
Requirement already up-to-date: b2 in /Library/Python/2.7/site-packages
Requirement already up-to-date: arrow>=0.8.0 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: logfury>=0.1.2 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: requests>=2.9.1 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: six>=1.10 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: tqdm>=4.5.0 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: futures>=3.0.5 in /Library/Python/2.7/site-packages (from b2)
Downloading/unpacking python-dateutil from https://pypi.python.org/packages/bc/c5/3449988d33baca4e9619f49a14e28026399b0a8c32817e28b503923a04ab/python_dateutil-2.7.0-py2.py3-none-any.whl#md5=5a86a548fe776cc079bf4a835473e3f8 (from arrow>=0.8.0->b2)
  Downloading python_dateutil-2.7.0-py2.py3-none-any.whl (207kB): 207kB downloaded
Installing collected packages: python-dateutil
  Found existing installation: python-dateutil 1.5
    Uninstalling python-dateutil:
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1431, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 598, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1836, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/util.py", line 295, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move
    copytree(src, real_dst, symlinks=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree
    raise Error, errors
Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil'")]

Storing debug log for failure in /Users/simon/Library/Logs/pip.log

I tried installing via the alternative method (with no luck)

git clone https://github.com/Backblaze/B2_Command_Line_Tool.git
Cloning into 'B2_Command_Line_Tool'...
remote: Counting objects: 5084, done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 5084 (delta 1), reused 1 (delta 0), pack-reused 5076
Receiving objects: 100% (5084/5084), 1.25 MiB | 689.00 KiB/s, done.
Resolving deltas: 100% (3622/3622), done.

cd B2_Command_Line_Tool/

I tried running the setup script (with no luck)

sudo python setup.py install
setuptools 20.2 or later is required. To fix, try running: pip install "setuptools>=20.2"

Upgrading setup tools also failed

sudo pip install "setuptools>=20.2"
Downloading/unpacking setuptools>=20.2
  Downloading setuptools-38.5.2-py2.py3-none-any.whl (490kB): 490kB downloaded
Installing collected packages: setuptools
  Found existing installation: setuptools 18.5
    Uninstalling setuptools:
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1431, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 598, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1836, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/util.py", line 295, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move
    copytree(src, real_dst, symlinks=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree
    raise Error, errors
Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib'")]

Storing debug log for failure in /Users/simon/Library/Logs/pip.log

Backing up a Mac via command line with B2

More to come when I can get B2 CLI Installed.

Backing up a an Ubuntu machine via command line with B2

More to come when I can get B2 CLI Installed.

Update

My ticket with Backblaze was automatically closed with this note “If the issue is persisting, it may be easiest to map the installation to the user folder, rather than the system level.”

No ideas how but something to research.

Ask a question or the recommend an article

[contact-form-7 404 "Not Found"]

Revision History

V1.1 ticket closed

v1.0 Initial post

Upgrading the RAM, CPU and Memory on a Vultr Ubuntu VM in the cloud

March 7, 2018 by Simon

Upgrading the RAM, CPU and Memory on a Vultr Ubuntu VM in the cloud is quite simple.

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. I prefer Vultr as they are located in the country (Australia) and are easy to use.

First, you need to shut down the server from within the VM (SSH), I used the command.

sudo shutdown now

Once the VM is shut down (wait a few minutes) you can turn off the VM in the Vultr GUI.

You can then go to Settings, Change Plan and review upgrade options.

Snapshot

Don’t forget to take a final snapshot.

Goto the Snapshots page (read this guide to restore a snapshot) and click Take Snapshot.

You can see snapshot progress on the main screen.

It may take a while for your snapshot to change from Pending to Processing.

Upgrade

When the snapshot is done it will auto boot and allow you to upgrade.

Choose the Upgrade specifications (Settings, Change Plan)

Click Upgrade

Confirm

The upgrade process will take a few minutes (I could see the CU and Ram was updated but the Storage was pending)

Testing

After the upgrade happened the VM will autoboot, login and check tour specifications (Useful Linux Commands).

I use the htop command to view specification information.

I did a quick benchmark pre-optimizing and I can see a speed bump of 0.2s. Time to optimize.

I threw 50 concurrent clients at my website (with loader.io) and the server handled it fine with no increase above memory capacity like before.

Optimize

Now I need to Optimize. Truth be told I did optimize and harden PHP and crashed PHP-FPM so I had o restore a VM snapshot.

Troubleshooting

If all else fails (post-upgrade configuration) you can restore the Vultr VM from a snapshot.

I hope this guide helps someone.

P.S If you don’t have a VM on Vultr click this link to set one up in minutes (setup guide here).

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.0 Initial post

How to upgrade a Digital Ocean Ubuntu VM and increase the vCPU or memory

February 6, 2018 by Simon

This blog post will show you how you can increase the memory and CPU allocation of an Ubuntu Server (Droplet) on Digital Ocean.

If you don’t have an Ubuntu server on Digital Ocean use this link ( https://m.do.co/c/99a5082b6de5 ) and get $10 free credit (2 months free). Read my guide here on setting it up.

Before you begin, ensure you have backed up your server. You can read here about setting up a new server on Digital Ocean from scratch, connecting to your server via SCP or automatically syncing files away from your server to another server with rsync .

In Jan 2018 Digital Ocean doubled the ram of $5/m servers from 512MB to 1GB so it’s time for me to get the free upgrade.

Connect to your server (via SSH or Web Console) and shut it down

shutdown -h now

After the server has shut down login to digital Ocean GUI and click (open) the server you want to upgrade.

Click Resize and choose the new upgraded server capacity

Click Resize (if the resize button is disabled you need to power off the server (via command line or via the power menu in Digital Ocean for the Droplet))

Click the Power On button under the Access tab when the resize is completed to restart the VM.

Congratulations, you will now have an upgraded server 🙂 Thank You Digital Ocean for the free RAM.

If you don’t have an Ubuntu server on Digital Ocean use this link ( https://m.do.co/c/99a5082b6de5 ) and get $10 free credit (2 months free). Read my guide here on setting it up.

Hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.0 Initial Post

Restoring a Vultr VM from a snapshot

January 26, 2018 by Simon

Ths is a short post showing how easy it is to restore a Vultr VM (Ubuntu) to a previous snapshot.

I have set up servers on Digital Ocean, AWS and Vultr before but it is always good to know how to restore a system when disaster strikes. Never rely on backups or snapshots to save you in the time of disaster but restoring a backup or snapshot will get you going to fast (depending on how frequent you backup). It is a good idea to know how to reconfigure your servers and also copy files and code to git or other offsite platforms regularly.

I recently had a test server fall over after upgrading MongoDB 3.4 to 3.6, I think I installed upstart to try and get MongoDB to run at startup.

My server won’t boot, bad sectors, invalid VM errors etc.

I tried repairing (based on advice around removing upstart etc)

But the suggested repair was for a GUI Ubuntu (and the server did not boot anyway).

3-week old snapshots available

Login to Vultr, click the server you want to restore, click Snapshots and then select the select the snapshot to restore and click the restore button on the right.

Restore will be started

Restore may take up to 1-hour ot more. Obviously, the server will be offline during the restore.

Restore progress will be available from the main server list.

You will be able to see when the snapshot has been restored.

You can verify the server is up by accessing the server (http or root console)

Don’t forget to take new snapshots regularly.

Consider automated backups

Now I just needed to follow my guide on instaling MongoDb 3.6 as this was the only change in the last 3 weeks (festive holidays).

Oh, and I had to patch for Spectre and Meltdown.

Yes you can restore a smaller VM onto a larger VM just not a larger snapshot onto a smaller,

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.2 Restoring a smaller snapshot onto a larger VM info

v1.1 Patch for Spectre and Meltdown

v1.0 Initial post

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View All Posts.

ubuntu

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4

Measuring VM disk performance on Ubuntu and comparing Vultr, Digital Ocean and UpCloud

Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX

Moving an Ubuntu 16.04 VM on Vultr from one data centre to another via snapshots

Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare

How to upgrade a Digital Ocean Ubuntu VM and increase the vCPU or memory

Restoring a Vultr VM from a snapshot

Popular

Security

Code

Tech

Wordpress

General

ubuntu

Footer

Popular

Security

Code

Tech

Wordpress

General