Code, InfoSec and Server Stuff
Views are personal and not my employers.
I recently had an issue where I set up a website for a friend. I invested 6 hours into setting up, the server and backups were automatically deleted after 7 days while I was away from my keyboard because I assumed the account was valid and had credits.
Read Full Article:
Here is how I added two subdomains (one pointing to a new UpCloud VM and the other pointing to an NGINX subsite) on Ubuntu 18.04
Advertisement:
If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.
UpCloud performance is great.
Buy a domain name from Namecheap here.
Goal(s)
Setup 2x subdomains on https://fearby.com
– Sub Domain #1: https://test.fearby.com (pointing to a dedicated UpCloud VM in Singapore for testing).
– Sub Domain #2: https://audit.fearby.com (pointing to a sub-website on the NGINX/VM that runs https://fearby.com )
Let’s set up the first Sub Domain (dedicated VM) and SSL
Backup
Do backup your server first.
VM
I created a second server ($5 month or $0.07c hour 1,024MB Memory, 25GB Disk, 1024 GB Month Data Transfer) at UpCloud. If you don’t already have an account at UpCloud use this link to signup and get $25 free credit ( https://www.upcloud.com/register/?promo=D84793 ). Read my blog post on why UpCloud is awesome and how I moved my domain to UpCloud.
Once I spun up a second server I obtained the IPv4 and IPv6 IP addresses of the new “test” VM from the UpCloud dashboard.
1 2 | IPV4 IP: 94.237.65.54 IPV6 IP: 2a04:3543:1000:2310:24b7:7cff:fe92:468c |
DNS
These DNS records were already in place with my DNS provider (Cloudflare).
1 2 | A fearby.com 209.50.48.88 AAAA fearby.com 2605:7380:1000:1310:24b7:7cff:fe92:0d64 |
I added these DNS records for the subdomains.
I added a new A NAME record for the new shared NGINX subdomain (for https://audit.fearby.com), this subdomain will be a sub-website that is running off the same server as https://fearby.com
1 2 | A audit 209.50.48.88 AAAA audit 2605:7380:1000:1310:24b7:7cff:fe92:0d64 |
I added another set of records for the new dedicated VM subdomain (for https://test.fearby.com)
1 2 | A test 94.237.65.54 AAAA test 2a04:3543:1000:2310:24b7:7cff:fe92:468c |
I waited for DNS to replicate around the globe by watching https://www.whatsmydns.net/
Setup a Firewall
On the new dedicated https://test.fearby.com VM, I installed the ufw firewall.
1 | sudo apt-get install ufw |
I configured the firewall to allow minimum ports (and added whitelisted IP for port 22 and added UpCloud DNS servers). I will lock this down some more later.
TIP: If your ISP does not offer a dedicated IP try a VPN. I use https://cyberghostvpn.com on OSX and Android.
Firewall rules.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | sudo ufw status numbered To Action From -- ------ ---- [ 1] 22 ALLOW IN x.x.x.x [ 2] 80 ALLOW IN Anywhere [ 3] 443 ALLOW IN Anywhere [ 4] 53 ALLOW IN 93.237.127.9 [ 5] 53 ALLOW IN 93.237.40.9 [ 6] 25 DENY IN Anywhere [ 7] 80 (v6) ALLOW IN Anywhere (v6) [ 8] 443 (v6) ALLOW IN Anywhere (v6) [ 9] 53 ALLOW IN 2a04:3540:53::1 [10] 53 ALLOW IN 2a04:3544:53::1 [11] 22 ALLOW IN x.x.x.x.x.x.x.x.x [12] 25 (v6) DENY IN Anywhere (v6) |
I enabled the firewall.
1 | sudo ufw enable |
Install NGINX (on https://test.fearby.com)
On the new dedicated https://test.fearby.com VM I…
Created a new www root
1 | mkdir /www-root |
Set permissions
1 | sudo chown -R www-data:www-data /www-root |
Installed NGINX
1 2 | sudo apt-get update sudo apt-get install nginx |
I created a placeholder webpage
1 | sudo nano /www-root/index.html |
Configured the root value in /etc/nginx/sites-available/default
Created a symbolic link of the nginx config
1 | sudo ln -s /etc/nginx/sites-available/default /etc/nginx/sites-enabled/default |
Lets Encrypt SSL
I have previously setup Lets encrypt on Ubuntu 16.04 but not 18.04. Certbot had info on setting up Lets Encrypt for 14.x 16.x and 17.x but not 18.x
Full credit for the SSL steps goes to @Linuxize ( tips on setting up Lets Encrypt on Ubuntu 18.04 ). Check out https://linuxize.com/
I installed Lets Encrypt certbot
1 2 | sudo apt update sudo apt install certbot |
I created a new Diffie–Hellman key
1 2 | mkdir -p /etc/ssl/certs/ sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048 |
Map requests to http://test.fearby.com/.well-known/acme-challenge to /var/lib/letsencrypt/.well-known ( Read the linuxize post for detailed steps ).
1 2 3 | mkdir -p /var/lib/letsencrypt/.well-known chgrp www-data /var/lib/letsencrypt chmod g+s /var/lib/letsencrypt |
Create a /etc/nginx/snippets/letsencrypt.conf on http://test.fearby.com and enforce the redirect.
1 2 3 4 5 6 | location ^~ /.well-known/acme-challenge/ { allow all; root /var/lib/letsencrypt/; default_type "text/plain"; try_files $uri =404; } |
Create a /etc/nginx/snippets/ssl.conf file on http://test.fearby.com
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | ssl_dhparam /etc/ssl/certs/dhparam.pem; ssl_session_timeout 1d; ssl_session_cache shared:SSL:50m; ssl_session_tickets off; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS'; ssl_prefer_server_ciphers on; ssl_stapling on; ssl_stapling_verify on; resolver 8.8.8.8 8.8.4.4 valid=300s; resolver_timeout 30s; add_header Strict-Transport-Security "max-age=15768000; includeSubdomains; preload"; add_header X-Frame-Options SAMEORIGIN; add_header X-Content-Type-Options nosniff; |
Let’s get a certificate
1 | sudo certbot certonly --agree-tos --email your@email.com --webroot -w /var/lib/letsencrypt/ -d test.fearby.com |
Certificates have been created 🙂
1 2 3 4 5 6 7 8 9 | ls -al /etc/letsencrypt/live/test.fearby.com/ total 12 drwxr-xr-x 2 user user 4096 Jun 26 11:30 . drwx------ 3 user user 4096 Jun 26 11:30 .. -rw-r--r-- 1 user user 543 Jun 26 11:30 README lrwxrwxrwx 1 user user 39 Jun 26 11:30 cert.pem -> ../../archive/test.fearby.com/cert1.pem lrwxrwxrwx 1 user user 40 Jun 26 11:30 chain.pem -> ../../archive/test.fearby.com/chain1.pem lrwxrwxrwx 1 user user 44 Jun 26 11:30 fullchain.pem -> ../../archive/test.fearby.com/fullchain1.pem lrwxrwxrwx 1 user user 42 Jun 26 11:30 privkey.pem -> ../../archive/test.fearby.com/privkey1.pem |
Now lets edit “/etc/nginx/sites-available/default” on https://test.fearby.com VM and add the cert paths.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | server { listen 80 default_server; listen [::]:80 default_server; listen 443 ssl http2; listen [::]:443 ssl http2; if ($scheme != "https") { return 301 https://$host$request_uri; } ssl_certificate /etc/letsencrypt/live/test.fearby.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/test.fearby.com/privkey.pem; ssl_trusted_certificate /etc/letsencrypt/live/test.fearby.com/chain.pem; include snippets/ssl.conf; #ssl_stapling on; # Requires nginx >= 1.3.7 # add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload"; add_header X-Frame-Options DENY; add_header X-Content-Type-Options nosniff; add_header X-XSS-Protection "1; mode=block"; root /www-root/; include snippets/letsencrypt.conf; index index.html; server_name test.fearby.com; location / { try_files $uri $uri/ =404; } } |
Advertisement:
Reload NGINX
1 | sudo systemctl reload nginx |
or
1 2 3 | sudo nginx -t sudo nginx -s reload sudo systemctl reload nginx |
Now let’s setup the second subdomain (subsite off https://fearby.com) and SSL
VM
I already have NGINX on https://fearby.com set up a second site.
DNS
We have already setup a DNS record for https://audit.fearby.com (above)
Firewall
Already configured at https://fearby.com
SSL
Because I had an existing Comodo certificate on https://fearby.com I am going to repeat the steps above to generate a new certificate but save the NGINX config to /etc/nginx/sites-available/audit.fearby.com (this activates the second site)
TIP: Follow the Linuxize guide here (for creating ssl.conf, letsencrypt.conf etc config files), Do a backup and restore if need be.
I created a new Diffie–Hellman key
1 2 | mkdir -p /etc/ssl/certs/ sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048 |
Let’s get a certificate
1 | sudo certbot certonly --agree-tos --email your@email.com --webroot -w /var/lib/letsencrypt/ -d audit.fearby.com |
Configure NGINX
Map requests to http://audit.fearby.com/.well-known/acme-challenge to /var/lib/letsencrypt/.well-known ( Read the linuxize post for detailed steps ).
1 2 3 | mkdir -p /var/lib/letsencrypt/.well-known chgrp www-data /var/lib/letsencrypt chmod g+s /var/lib/letsencrypt |
I created a new NGINX site ( /etc/nginx/sites-available/audit.fearby.com )
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 | #proxy_cache_path /tmp/nginx-cache keys_zone=one:10m;# server { root /www-audit-root; # Listen Ports listen 80; listen [::]:80; listen 443 ssl http2; listen [::]:443 ssl http2; # Default File index index.html index.php index.htm; # Server Name server_name audit.fearby.com; include snippets/letsencrypt.conf; location / { try_files $uri $uri/ =404; } ssl_certificate /etc/letsencrypt/live/audit.fearby.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/audit.fearby.com/privkey.pem; ssl_trusted_certificate /etc/letsencrypt/live/audit.fearby.com/chain.pem; ssl_dhparam /etc/ssl/certs/auditdhparam.pem; ssl_session_timeout 1d; #ssl_session_cache shared:SSL:50m; ssl_session_tickets off; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA38$ ssl_prefer_server_ciphers on; ssl_stapling on; ssl_stapling_verify on; #resolver 8.8.8.8 8.8.4.4 valid=300s; #resolver_timeout 30s; add_header Strict-Transport-Security "max-age=15768000; includeSubdomains; preload"; add_header X-Frame-Options SAMEORIGIN; add_header X-Content-Type-Options nosniff; if ($scheme != "https") { return 301 https://$host$request_uri; } } |
I created a symbolic link of the config file
1 | sudo ln -s /etc/nginx/sites-available/audit.fearby.com /etc/nginx/sites-enabled/audit.fearby.com |
Reload NGINX
1 | sudo systemctl reload nginx |
or
1 2 3 | sudo nginx -t sudo nginx -s reload sudo systemctl reload nginx |
How to test the certificate renewal
1 | sudo certbot renew --dry-run |
Automate the renewal in crontab (every 12 hours)
I set this crontab entry up on https://fearby.com and https://test.fearby.com
1 2 | crontab -e 0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(3600))' && certbot -q renew --renew-hook "systemctl reload nginx" |
Conclusion
Yes, I haVe 2 subdomains (1x dedicated VM and the other is a sub-website off an existing server) with SSL certificates.
Ping Results
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | ping -c 4 fearby.com PING fearby.com (209.50.48.88): 56 data bytes 64 bytes from 209.50.48.88: icmp_seq=0 ttl=44 time=220.000 ms 64 bytes from 209.50.48.88: icmp_seq=1 ttl=44 time=290.602 ms 64 bytes from 209.50.48.88: icmp_seq=2 ttl=44 time=311.938 ms 64 bytes from 209.50.48.88: icmp_seq=3 ttl=44 time=330.841 ms --- fearby.com ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 220.000/288.345/330.841/41.948 ms ping -c 4 test.fearby.com PING test.fearby.com (94.237.65.54): 56 data bytes 64 bytes from 94.237.65.54: icmp_seq=0 ttl=44 time=333.590 ms 64 bytes from 94.237.65.54: icmp_seq=1 ttl=44 time=252.433 ms 64 bytes from 94.237.65.54: icmp_seq=2 ttl=44 time=271.153 ms 64 bytes from 94.237.65.54: icmp_seq=3 ttl=44 time=292.685 ms --- test.fearby.com ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 252.433/287.465/333.590/30.200 ms ping -c 4 audit.fearby.com PING audit.fearby.com (209.50.48.88): 56 data bytes 64 bytes from 209.50.48.88: icmp_seq=0 ttl=44 time=281.662 ms 64 bytes from 209.50.48.88: icmp_seq=1 ttl=44 time=307.676 ms 64 bytes from 209.50.48.88: icmp_seq=2 ttl=44 time=227.985 ms 64 bytes from 209.50.48.88: icmp_seq=3 ttl=44 time=215.566 ms --- audit.fearby.com ping statistics --- 4 packets transmitted, 4 packets received, 0.0% packet loss round-trip min/avg/max/stddev = 215.566/258.222/307.676/37.845 ms |
Webpage Results
Advertisement:
I hope this guide helps someone.
Please consider using my referral code and get $25 credit for free.
https://www.upcloud.com/register/?promo=D84793
Ask a question or recommend an article
Revision History
v1.0 Initial Post
Before UpCloud, when I had 512MB ram on Vultr I had multiple NGINX crashed a day so I used a bash script and scheduled a cron job to clear memory cache when memory fell below 100MB (view the script here). To further increase the speed of the WordPress I have configured the OS to use memory over the disk. About once a day free memory falls below 100MB (this is not a problem as my script clears cached items automatically).
Read the full article here: https://fearby.com/article/upgrading-an-ubuntu-server-on-upcloud-to-add-more-cpu-memory-and-disk-space/
Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space
Advertisement:
If you have not read my previous posts I have now moved my blog from Vultr to the awesome UpCloud host (signup using this link to get $25 free credit).
Recently I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.
Spoiler: UpCloud performance is great.
Why Upgrade
I have 1 CPU, 1 GB memory and 50GB storage and it is running well? I have PHP child workers set up and have set up the preferred use of memory over swap file usage.
Before UpCloud, when I had 512MB ram on Vultr I had multiple NGINX crashed a day so I used a bash script and scheduled a cron job to clear memory cache when memory fell below 100MB (view the script here). To further increase the speed of the WordPress I have configured the OS to use memory over the disk. About once a day free memory falls below 100MB (this is not a problem as my script clears cached items automatically).
I’d like to add more memory as I am working on some things (watch this space) and I will use the extra memory. I’d prefer the server is set up now for the expected workload.
How to Upgrade
This is how I upgraded from 1xCPU/2GB Memory/50GB Storage/2TB Transferred Data to a 2 CPU/4GB Mmeory/80GB Storage/4TB Transferred Data server.
UpCloud Pricing: https://www.upcloud.com/pricing/
Upgrade an UpCloud VM
I shut down my existing VM. Read this guide to setup a VM.
1 | shutdown -P |
Login to the UpCloud dashboard, select your server (confirm the server has shut down) and click General Settings, choose the upgrade and click Update.
I confirmed the upgrade options (2x CPU, 4096 MB Memory).
Click Update
After 10 seconds you can start your server from the UpCloud server.
I confirmed the CPU upgrade was visible in the VM
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 63 model name : Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz stepping : 2 microcode : 0x1 cpu MHz : 3099.978 cache size : 16384 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass bogomips : 6199.95 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 63 model name : Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz stepping : 2 microcode : 0x1 cpu MHz : 3099.978 cache size : 16384 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 apicid : 1 initial apicid : 1 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass bogomips : 6199.95 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: |
Software Tweaks Post Upgrade.
I added these settings to the top of /etc/nginx/nginx.conf to ensure the extra CPU was used.
1 2 | worker_processes auto; worker_cpu_affinity auto; |
I increased PHP FPM ( /etc/php/7.2/fpm/php.ini ) to increase the memory usage and child workers. I doubled child workers and max memeory limit.
1 2 3 4 5 | memory_limit = 3072M pm.max_children = 80 pm.start_servers = 30 pm.min_spare_servers = 10 pm.max_spare_servers = 30 |
I restarted NGINX and PHP
1 2 3 4 | nginx -t nginx -s reload /etc/init.d/nginx restart service php7.2-fpm restart |
I tweaked WordPress max memory limits
1 2 | define( 'WP_MEMORY_LIMIT','3072M'); define( 'WP_MAX_MEMORY_LIMIT','3072M'); |
MySQL Tweaks: I logged into MySQL
1 | mysql -u root -p |
I ran “SHOW GLOBAL STATUS” to view stats
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 | mysql> SHOW GLOBAL STATUS; +-----------------------------------------------+--------------------------------------------------+ | Variable_name | Value | +-----------------------------------------------+--------------------------------------------------+ | Aborted_clients | 0 | | Aborted_connects | 0 | | Binlog_cache_disk_use | 0 | | Binlog_cache_use | 0 | | Binlog_stmt_cache_disk_use | 0 | | Binlog_stmt_cache_use | 0 | | Bytes_received | 3179986 | | Bytes_sent | 223872114 | | Com_admin_commands | 0 | | Com_assign_to_keycache | 0 | | Com_alter_db | 0 | | Com_alter_db_upgrade | 0 | | Com_alter_event | 0 | | Com_alter_function | 0 | | Com_alter_instance | 0 | | Com_alter_procedure | 0 | | Com_alter_server | 0 | | Com_alter_table | 0 | | Com_alter_tablespace | 0 | | Com_alter_user | 0 | | Com_analyze | 0 | | Com_begin | 0 | | Com_binlog | 0 | | Com_call_procedure | 0 | | Com_change_db | 284 | | Com_change_master | 0 | | Com_change_repl_filter | 0 | | Com_check | 0 | | Com_checksum | 0 | | Com_commit | 0 | | Com_create_db | 0 | | Com_create_event | 0 | | Com_create_function | 0 | | Com_create_index | 0 | | Com_create_procedure | 0 | | Com_create_server | 0 | | Com_create_table | 0 | | Com_create_trigger | 0 | | Com_create_udf | 0 | | Com_create_user | 0 | | Com_create_view | 0 | | Com_dealloc_sql | 0 | | Com_delete | 18 | | Com_delete_multi | 0 | | Com_do | 0 | | Com_drop_db | 0 | | Com_drop_event | 0 | | Com_drop_function | 0 | | Com_drop_index | 0 | | Com_drop_procedure | 0 | | Com_drop_server | 0 | | Com_drop_table | 0 | | Com_drop_trigger | 0 | | Com_drop_user | 0 | | Com_drop_view | 0 | | Com_empty_query | 0 | | Com_execute_sql | 0 | | Com_explain_other | 0 | | Com_flush | 0 | | Com_get_diagnostics | 0 | | Com_grant | 0 | | Com_ha_close | 0 | | Com_ha_open | 0 | | Com_ha_read | 0 | | Com_help | 0 | | Com_insert | 342 | | Com_insert_select | 0 | | Com_install_plugin | 0 | | Com_kill | 0 | | Com_load | 0 | | Com_lock_tables | 0 | | Com_optimize | 0 | | Com_preload_keys | 0 | | Com_prepare_sql | 0 | | Com_purge | 0 | | Com_purge_before_date | 0 | | Com_release_savepoint | 0 | | Com_rename_table | 0 | | Com_rename_user | 0 | | Com_repair | 0 | | Com_replace | 0 | | Com_replace_select | 0 | | Com_reset | 0 | | Com_resignal | 0 | | Com_revoke | 0 | | Com_revoke_all | 0 | | Com_rollback | 0 | | Com_rollback_to_savepoint | 0 | | Com_savepoint | 0 | | Com_select | 16358 | | Com_set_option | 849 | | Com_signal | 0 | | Com_show_binlog_events | 0 | | Com_show_binlogs | 0 | | Com_show_charsets | 0 | | Com_show_collations | 0 | | Com_show_create_db | 0 | | Com_show_create_event | 0 | | Com_show_create_func | 0 | | Com_show_create_proc | 0 | | Com_show_create_table | 0 | | Com_show_create_trigger | 0 | | Com_show_databases | 3 | | Com_show_engine_logs | 0 | | Com_show_engine_mutex | 0 | | Com_show_engine_status | 0 | | Com_show_events | 0 | | Com_show_errors | 0 | | Com_show_fields | 921 | | Com_show_function_code | 0 | | Com_show_function_status | 0 | | Com_show_grants | 0 | | Com_show_keys | 1 | | Com_show_master_status | 0 | | Com_show_open_tables | 0 | | Com_show_plugins | 0 | | Com_show_privileges | 0 | | Com_show_procedure_code | 0 | | Com_show_procedure_status | 0 | | Com_show_processlist | 0 | | Com_show_profile | 0 | | Com_show_profiles | 0 | | Com_show_relaylog_events | 0 | | Com_show_slave_hosts | 0 | | Com_show_slave_status | 0 | | Com_show_status | 6 | | Com_show_storage_engines | 0 | | Com_show_table_status | 0 | | Com_show_tables | 2 | | Com_show_triggers | 0 | | Com_show_variables | 6 | | Com_show_warnings | 1 | | Com_show_create_user | 0 | | Com_shutdown | 0 | | Com_slave_start | 0 | | Com_slave_stop | 0 | | Com_group_replication_start | 0 | | Com_group_replication_stop | 0 | | Com_stmt_execute | 4 | | Com_stmt_close | 4 | | Com_stmt_fetch | 0 | | Com_stmt_prepare | 4 | | Com_stmt_reset | 0 | | Com_stmt_send_long_data | 4 | | Com_truncate | 2 | | Com_uninstall_plugin | 0 | | Com_unlock_tables | 0 | | Com_update | 70 | | Com_update_multi | 0 | | Com_xa_commit | 0 | | Com_xa_end | 0 | | Com_xa_prepare | 0 | | Com_xa_recover | 0 | | Com_xa_rollback | 0 | | Com_xa_start | 0 | | Com_stmt_reprepare | 0 | | Connection_errors_accept | 0 | | Connection_errors_internal | 0 | | Connection_errors_max_connections | 0 | | Connection_errors_peer_address | 0 | | Connection_errors_select | 0 | | Connection_errors_tcpwrap | 0 | | Connections | 292 | | Created_tmp_disk_tables | 1124 | | Created_tmp_files | 30 | | Created_tmp_tables | 1369 | | Delayed_errors | 0 | | Delayed_insert_threads | 0 | | Delayed_writes | 0 | | Flush_commands | 1 | | Handler_commit | 6094 | | Handler_delete | 33 | | Handler_discover | 0 | | Handler_external_lock | 38571 | | Handler_mrr_init | 0 | | Handler_prepare | 0 | | Handler_read_first | 2299 | | Handler_read_key | 134761 | | Handler_read_last | 237 | | Handler_read_next | 310119 | | Handler_read_prev | 2733 | | Handler_read_rnd | 222350 | | Handler_read_rnd_next | 472820 | | Handler_rollback | 0 | | Handler_savepoint | 0 | | Handler_savepoint_rollback | 0 | | Handler_update | 15605 | | Handler_write | 17310 | | Innodb_buffer_pool_dump_status | Dumping of buffer pool not started | | Innodb_buffer_pool_load_status | Buffer pool(s) load completed at 180624 23:38:01 | | Innodb_buffer_pool_resize_status | | | Innodb_buffer_pool_pages_data | 1035 | | Innodb_buffer_pool_bytes_data | 16957440 | | Innodb_buffer_pool_pages_dirty | 0 | | Innodb_buffer_pool_bytes_dirty | 0 | | Innodb_buffer_pool_pages_flushed | 1936 | | Innodb_buffer_pool_pages_free | 7144 | | Innodb_buffer_pool_pages_misc | 13 | | Innodb_buffer_pool_pages_total | 8192 | | Innodb_buffer_pool_read_ahead_rnd | 0 | | Innodb_buffer_pool_read_ahead | 0 | | Innodb_buffer_pool_read_ahead_evicted | 0 | | Innodb_buffer_pool_read_requests | 306665 | | Innodb_buffer_pool_reads | 950 | | Innodb_buffer_pool_wait_free | 0 | | Innodb_buffer_pool_write_requests | 26509 | | Innodb_data_fsyncs | 1229 | | Innodb_data_pending_fsyncs | 0 | | Innodb_data_pending_reads | 0 | | Innodb_data_pending_writes | 0 | | Innodb_data_read | 16273920 | | Innodb_data_reads | 1078 | | Innodb_data_writes | 2857 | | Innodb_data_written | 53379584 | | Innodb_dblwr_pages_written | 1275 | | Innodb_dblwr_writes | 109 | | Innodb_log_waits | 0 | | Innodb_log_write_requests | 450 | | Innodb_log_writes | 585 | | Innodb_os_log_fsyncs | 793 | | Innodb_os_log_pending_fsyncs | 0 | | Innodb_os_log_pending_writes | 0 | | Innodb_os_log_written | 664064 | | Innodb_page_size | 16384 | | Innodb_pages_created | 56 | | Innodb_pages_read | 988 | | Innodb_pages_written | 1936 | | Innodb_row_lock_current_waits | 0 | | Innodb_row_lock_time | 0 | | Innodb_row_lock_time_avg | 0 | | Innodb_row_lock_time_max | 0 | | Innodb_row_lock_waits | 0 | | Innodb_rows_deleted | 2 | | Innodb_rows_inserted | 19219 | | Innodb_rows_read | 249102 | | Innodb_rows_updated | 77 | | Innodb_num_open_files | 81 | | Innodb_truncated_status_writes | 0 | | Innodb_available_undo_logs | 128 | | Key_blocks_not_flushed | 0 | | Key_blocks_unused | 12751 | | Key_blocks_used | 645 | | Key_read_requests | 321877 | | Key_reads | 648 | | Key_write_requests | 196 | | Key_writes | 150 | | Locked_connects | 0 | | Max_execution_time_exceeded | 0 | | Max_execution_time_set | 0 | | Max_execution_time_set_failed | 0 | | Max_used_connections | 3 | | Max_used_connections_time | 2018-06-24 23:43:48 | | Not_flushed_delayed_rows | 0 | | Ongoing_anonymous_transaction_count | 0 | | Open_files | 229 | | Open_streams | 0 | | Open_table_definitions | 206 | | Open_tables | 786 | | Opened_files | 502 | | Opened_table_definitions | 208 | | Opened_tables | 817 | | Performance_schema_accounts_lost | 0 | | Performance_schema_cond_classes_lost | 0 | | Performance_schema_cond_instances_lost | 0 | | Performance_schema_digest_lost | 0 | | Performance_schema_file_classes_lost | 0 | | Performance_schema_file_handles_lost | 0 | | Performance_schema_file_instances_lost | 0 | | Performance_schema_hosts_lost | 0 | | Performance_schema_index_stat_lost | 0 | | Performance_schema_locker_lost | 0 | | Performance_schema_memory_classes_lost | 0 | | Performance_schema_metadata_lock_lost | 0 | | Performance_schema_mutex_classes_lost | 0 | | Performance_schema_mutex_instances_lost | 0 | | Performance_schema_nested_statement_lost | 0 | | Performance_schema_prepared_statements_lost | 0 | | Performance_schema_program_lost | 0 | | Performance_schema_rwlock_classes_lost | 0 | | Performance_schema_rwlock_instances_lost | 0 | | Performance_schema_session_connect_attrs_lost | 0 | | Performance_schema_socket_classes_lost | 0 | | Performance_schema_socket_instances_lost | 0 | | Performance_schema_stage_classes_lost | 0 | | Performance_schema_statement_classes_lost | 0 | | Performance_schema_table_handles_lost | 0 | | Performance_schema_table_instances_lost | 0 | | Performance_schema_table_lock_stat_lost | 0 | | Performance_schema_thread_classes_lost | 0 | | Performance_schema_thread_instances_lost | 0 | | Performance_schema_users_lost | 0 | | Prepared_stmt_count | 0 | | Qcache_free_blocks | 1 | | Qcache_free_memory | 16760152 | | Qcache_hits | 0 | | Qcache_inserts | 0 | | Qcache_lowmem_prunes | 0 | | Qcache_not_cached | 16355 | | Qcache_queries_in_cache | 0 | | Qcache_total_blocks | 1 | | Queries | 19164 | | Questions | 19155 | | Select_full_join | 0 | | Select_full_range_join | 0 | | Select_range | 2677 | | Select_range_check | 0 | | Select_scan | 2098 | | Slave_open_temp_tables | 0 | | Slow_launch_threads | 0 | | Slow_queries | 0 | | Sort_merge_passes | 12 | | Sort_range | 4859 | | Sort_rows | 244452 | | Sort_scan | 854 | | Ssl_accept_renegotiates | 0 | | Ssl_accepts | 0 | | Ssl_callback_cache_hits | 0 | | Ssl_cipher | | | Ssl_cipher_list | | | Ssl_client_connects | 0 | | Ssl_connect_renegotiates | 0 | | Ssl_ctx_verify_depth | 0 | | Ssl_ctx_verify_mode | 0 | | Ssl_default_timeout | 0 | | Ssl_finished_accepts | 0 | | Ssl_finished_connects | 0 | | Ssl_server_not_after | | | Ssl_server_not_before | | | Ssl_session_cache_hits | 0 | | Ssl_session_cache_misses | 0 | | Ssl_session_cache_mode | NONE | | Ssl_session_cache_overflows | 0 | | Ssl_session_cache_size | 0 | | Ssl_session_cache_timeouts | 0 | | Ssl_sessions_reused | 0 | | Ssl_used_session_cache_entries | 0 | | Ssl_verify_depth | 0 | | Ssl_verify_mode | 0 | | Ssl_version | | | Table_locks_immediate | 11962 | | Table_locks_waited | 0 | | Table_open_cache_hits | 19395 | | Table_open_cache_misses | 817 | | Table_open_cache_overflows | 12 | | Tc_log_max_pages_used | 0 | | Tc_log_page_size | 0 | | Tc_log_page_waits | 0 | | Threads_cached | 2 | | Threads_connected | 1 | | Threads_created | 3 | | Threads_running | 1 | | Uptime | 2944 | | Uptime_since_flush_status | 2944 | +-----------------------------------------------+--------------------------------------------------+ 353 rows in set (0.00 sec) |
Read more on SHOW GLOBAL STATUS here. Read more on the values here.
I can see NO major errors here (possibly due to UpClouds awesome disk IO) so I won’t be making memory tweaks in MySQL. Sign Up using this link and get $25 credit free on UpCloud and see for yourself how fast they are.
Configure Ubuntu System Memory Usage
Edit /etc/sysctl.conf
Add the following to allow things to sit in ram longer
1 | vm.vfs_cache_pressure=50 |
Snip from: https://www.kernel.org/doc/Documentation/sysctl/vm.txt
1 2 3 4 5 | This percentage value controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects. At the default value of vfs_cache_pressure=100 the kernel will attempt to reclaim dentries and inodes at a "fair" rate with respect to pagecache and swapcache reclaim. Decreasing vfs_cache_pressure causes the kernel to prefer to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will never reclaim dentries and inodes due to memory pressure and this can easily lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100 causes the kernel to prefer to reclaim dentries and inodes. Increasing vfs_cache_pressure significantly beyond 100 may have negative performance impact. Reclaim code needs to take various locks to find freeable directory and inode objects. With vfs_cache_pressure=1000, it will look for ten times more freeable objects than there are." |
Read these pages here and here regarding setting MySQL memory.
Reboot
1 | shutdown -r now |
Resize the disk
The 2x CPU, 4GB memory plan comes with 80GB storage allowance. My disk at present is 50GB and I will update the size soon following this guide soon.
Quick Benchmark
I used loader.io to load 500 users to access my site in 1 minute.
The benchmark worked with no errors.
This benchmark was performed with no Cloudflare caching. I should get Cloudflare caching working again to lower the average response time. I loaded my website manually in Google Chrome while loader.io threw 500 users at my site and it loaded very fast.
Conclusion
After a few days, I checked my memory logs and there were no low memory triggers (just normal internal memory management triggers). Ubuntu was happier.
This graph was taken before I set “vm.vfs_cache_pressure” so I will update this graph in a few days.
I hope this guide helps someone.
Please consider using my referral code and get $25 credit for free.
https://www.upcloud.com/register/?promo=D84793
Ask a question or recommend an article
Revision History
1.0 Initial Post
How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 4 of 4
Advertisement:
Read Part 1, Part 2, Part 3 or Part 4
I ran the MySQL benchmark preparation command again (no problem this time).
1 2 3 4 5 | sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=###################### prepare sysbench 0.4.12: multi-threaded system evaluation benchmark Creating table 'sbtest'... Creating 1000000 records in table 'sbtest'... |
Test table and records created
Now I can benchmark MySQL on my main server.
1 | sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=################################# --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run |
RAW Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 8 Doing OLTP test. Running mixed OLTP test Doing read-only test Using Special distribution (12 iterations, 1 pct of values are returned in 75 pct cases) Using "BEGIN" for starting transactions Using auto_inc on the id column Threads started! Time limit exceeded, exiting... (last message repeated 7 times) Done. OLTP test statistics: queries performed: read: 336210 write: 0 other: 48030 total: 384240 transactions: 24015 (400.09 per sec.) deadlocks: 0 (0.00 per sec.) read/write requests: 336210 (5601.24 per sec.) other operations: 48030 (800.18 per sec.) Test execution summary: total time: 60.0242s total number of events: 24015 total time taken by event execution: 480.0242 per-request statistics: min: 1.79ms avg: 19.99ms max: 141.00ms approx. 95 percentile: 37.49ms Threads fairness: events (avg/stddev): 3001.8750/27.36 execution time (avg/stddev): 60.0030/0.01 |
Results
queries performed (in 60 seconds):
I decided to add an index to see if I can speed this query up (read the MySQL index page here). I added an index (in Adminer) on the columns “Id” and “pad” for the sbtest table in the test database
I restarted the MySQL process
1 2 | mysql restart [ ok ] Restarting mysql (via systemctl): mysql.service. |
I ran the same benchmark again.
Raw Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=########################## --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 8 Doing OLTP test. Running mixed OLTP test Doing read-only test Using Special distribution (12 iterations, 1 pct of values are returned in 75 pct cases) Using "BEGIN" for starting transactions Using auto_inc on the id column Threads started! Time limit exceeded, exiting... (last message repeated 7 times) Done. OLTP test statistics: queries performed: read: 426538 write: 0 other: 60934 total: 487472 transactions: 30467 (507.69 per sec.) deadlocks: 0 (0.00 per sec.) read/write requests: 426538 (7107.67 per sec.) other operations: 60934 (1015.38 per sec.) Test execution summary: total time: 60.0110s total number of events: 30467 total time taken by event execution: 479.9124 per-request statistics: min: 5.75ms avg: 15.75ms max: 138.57ms approx. 95 percentile: 25.10ms Threads fairness: events (avg/stddev): 3808.3750/8.70 execution time (avg/stddev): 59.9891/0.00 |
Results
The quick index added 20% extra throughput on queries 🙂
Don’t forget to delete your test database
1 | DROP DATABASE `test`; |
Viewing MySQL Index Usage (on the “test” database)
Query to show Index stats for a table ‘test’
1 2 3 4 5 6 7 8 9 10 11 12 | SELECT OBJECT_SCHEMA as 'Database', OBJECT_NAME as 'Table', INDEX_NAME as 'Index', COUNT_STAR, SUM_TIMER_WAIT, MIN_TIMER_WAIT, AVG_TIMER_WAIT, MAX_TIMER_WAIT, COUNT_READ, SUM_TIMER_READ, MIN_TIMER_READ, AVG_TIMER_READ, MAX_TIMER_READ, COUNT_FETCH, SUM_TIMER_FETCH, MIN_TIMER_FETCH, AVG_TIMER_FETCH, MAX_TIMER_FETCH FROM performance_schema.table_io_waits_summary_by_index_usage WHERE object_schema = 'test' |
I can see the MySQL PRIMARY index is getting used 🙂
Read more in viewable query stats (columns) here.
Other System Information Tools
Show processor information
1 | cat /proc/cpuinfo |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 61 model name : Virtual CPU a7769a6388d5 stepping : 2 microcode : 0x1 cpu MHz : 2394.454 cache size : 16384 KB physical id : 0 siblings : 1 core id : 0 cpu cores : 1 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single kaiser fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass bogomips : 4788.90 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: |
Memory Information
You can assign 512MB, 1GB, 2GB or more memory to a server on Vultr, Read my guide here on upgrading resources for Vultr VM’s here.
Only upgrade your server’s memory when server processes demand it, there is no need to pay for extra idle memory. Read my older guides on upgrading Digital Ocean and AWS servers.
I use the htop utility to monitor memory and processes. The memory usage will depend on how you have configured your server to use connection pools in code, MySQL or services. Also what memory demands do you get in pean bandwidth times?
You can check your server memory details on Ubuntu with this command
1 | cat /proc/meminfo |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | MemTotal: 2048104 kB MemFree: 96176 kB MemAvailable: 693072 kB Buffers: 183476 kB Cached: 526124 kB SwapCached: 0 kB Active: 1467220 kB Inactive: 243228 kB Active(anon): 1070464 kB Inactive(anon): 27004 kB Active(file): 396756 kB Inactive(file): 216224 kB Unevictable: 3652 kB Mlocked: 3652 kB SwapTotal: 0 kB SwapFree: 0 kB Dirty: 64 kB Writeback: 0 kB AnonPages: 1004504 kB Mapped: 114664 kB Shmem: 94192 kB Slab: 192692 kB SReclaimable: 171892 kB SUnreclaim: 20800 kB KernelStack: 3072 kB PageTables: 20528 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 1024052 kB Committed_AS: 2424332 kB VmallocTotal: 34359738367 kB VmallocUsed: 0 kB VmallocChunk: 0 kB HardwareCorrupted: 0 kB AnonHugePages: 247808 kB CmaTotal: 0 kB CmaFree: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB DirectMap4k: 67440 kB DirectMap2M: 2029568 kB |
Use Memory or Disk (Swap)
You can configure the use of Memory over Disk by configuring your/etc/sysctl.conf file (setting value “vm.swappiness”)
You can check your swap file settings by running the following command
1 2 | cat /proc/sys/vm/swappiness 1 |
Or By running
1 2 | sysctl vm.swappiness vm.swappiness = 1 |
Set a new swap file value by editing /etc/sysctl.conf
1 | sudo nano /etc/sysctl.conf |
Set the following to use more ram over the swap disk.
1 | vm.swappiness = 1 |
Read about swappiness values here: https://en.wikipedia.org/wiki/Swappiness
Service Performance
Performance (and allocated resources) depends on the demands of your operating system and installed software
What operating system do you have?
1 2 3 4 5 6 | lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 16.04.4 LTS Release: 16.04 Codename: xenial |
View NGINX Status, how much memory does it use?
1 2 3 4 5 6 7 8 9 10 11 12 13 | /etc/init.d/nginx status ● nginx.service - A high performance web server and a reverse proxy server Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2018-05-25 21:28:25 AEST; 1 weeks 3 days ago Docs: man:nginx(8) Main PID: #### (nginx) Tasks: 3 Memory: 58.9M CPU: 33min 11.515s CGroup: /system.slice/nginx.service ├─#### nginx: master process /usr/sbin/nginx -g daemon on; master_process on; ├─#### nginx: worker process └─#### nginx: cache manager process |
PHP (and Child Worker) status how much memory does it use and how many child workers do you have? Read my add PHP child workers post here (and update to PHP 7.2 here)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | sudo service php7.2-fpm status ● php7.2-fpm.service - The PHP 7.2 FastCGI Process Manager Loaded: loaded (/lib/systemd/system/php7.2-fpm.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2018-05-25 21:28:26 AEST; 1 weeks 3 days ago Docs: man:php-fpm7.2(8) Main PID: #### (php-fpm7.2) Status: "Processes active: 0, idle: 20, Requests: 75911, slow: 0, Traffic: 0.1req/sec" Tasks: 21 Memory: 694.2M CPU: 20h 49min 45.132s CGroup: /system.slice/php7.2-fpm.service ├─ #### php-fpm: master process (/etc/php/7.2/fpm/php-fpm.conf) ├─ #### php-fpm: pool www-acc ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-acc ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr ├─ #### php-fpm: pool www-usr └─ #### php-fpm: pool www-usr |
MySQL Status
1 2 3 4 5 6 7 8 9 10 | sudo service mysql status ● mysql.service - MySQL Community Server Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2018-05-25 21:28:27 AEST; 1 weeks 3 days ago Main PID: ##### (mysqld) Tasks: 35 Memory: 405.9M CPU: 2h 17min 31.822s CGroup: /system.slice/mysql.service └─#### /usr/sbin/mysqld |
Shared VM Hosts
One of the biggest impacts (after server latency) for your server is not the disk performance but the number of hosts/websites on the server who are also using the disk and server resources.
I have 80 other web servers on my server (based on a reverse lookup).
I may move to a dedicated box when I can afford it.
Security
Above all else ensure that security is number 1 priority and make performance second priority.
Scan your site with Zap, Qualys and Kali Linux. Performance means nothing if you are hacked.
Simulated concurrent users
You can use Siege to test the maximum concurrent users accessing your site before the server starts to drop connections.
FYI: If you use Cloudflare (you should) this may not work as it will block connections.
Install Siege
1 | sudo apt-get install siege |
Test your server with 10 concurrent serves for 1 minute
1 | siege -t1m c10 'https://yourserver.com/' |
Results
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | siege -t1m c10 'https://yourserver.com/' ** SIEGE 3.0.8 ** Preparing 15 concurrent users for battle. The server is now under siege... Lifting the server siege... done. Transactions: 417 hits Availability: 100.00 % Elapsed time: 59.01 secs Data transferred: 8.24 MB Response time: 1.62 secs Transaction rate: 7.07 trans/sec Throughput: 0.14 MB/sec Concurrency: 11.46 Successful transactions: 417 Failed transactions: 0 Longest transaction: 2.26 Shortest transaction: 1.49 |
Keep upping the connections (from 10 above) to a limit where connections start dropping.
I tried 25 then 50 concurrent users hitting a server on Digital Ocean and it did not fail.
Conclusion
Create your own server today
And remember you can install the Runcloud server management dashboard here.
I hope this guide helps someone.
How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 3 of 4
Advertisement:
Read Part 1, Part 2, Part 3 or Part 4
I used these commands to generate bonnie++ reports from the data in part 2
1 2 3 4 5 6 7 | echo "<h1>Bonnie Results</h1>" > /www-data/bonnie.html echo "<h2>Vultr (Sydney)</h2>" >> /www-data/bonnie.html echo "1.97,1.97,servername,1,1528177870,4G,,656,99,308954,68,113706,33,1200,92,188671,30,10237,251,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,26067us,119ms,179ms,29139us,26069us,16118us,1463us,703us,880us,263us,119us,593us" | bon_csv2html >> /www-data/bonnie.html echo "<h2>Digital Ocean (London)</h2>" >> /www-data/bonnie.html echo "1.97,1.97,servername,1,1528186398,4G,,699,99,778636,74,610414,60,1556,99,1405337,59,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,17678us,10099us,17014us,7027us,3067us,2366us,1243us,376us,611us,108us,59us,181us" | bon_csv2html >> /www-data/bonnie.html echo "<h2>UpCloud (Singapore)</h2>" >> /www-data/bonnie.html echo "1.97,1.97,servername,1,1528226703,4G,,1014,99,407179,24,366622,32,2137,99,451886,17,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,11297us,54232us,16443us,4949us,44883us,1595us,264us,340us,561us,138us,66us,327us" | bon_csv2html >> /www-data/bonnie.html |
Image of results here
Network Performace
IMHO Network Latency is the biggest impact on server performance, Read my old post on scalability on a budget here. I am in Australia an having a server in Singapore was too far away and latency was terrible.
Here is a non-scientific example of pinging a Vultr, Digital Ocean and UpCloud server in three different locations (and Google).
Test Ping Results
Latency matters, run a https://www.webpagetest.org/ scan over your site to see why.
Adding https added almost 0.7 seconds to https communications in the past on Digital Ocean (a few thousand kilometres away). The longer the latency the longer HTTPS handshakes take.
Deploying a server to Singapore (in my experience) is bad if your visitors are in Australia. But deploying to other regions may be lower in cost though. It’s a trade off.
Deploy servers as close as you can to your customers is the best tip for performance.
Also, consider setting up Image Optimization and Image CDN plugins (guide here) in WordPress and using Cloudflare (guide here)
Benchmarking with SysBench
Install CPU Benchmark
1 | sudo apt-get install sysbench |
CPU Benchmark (Vultr/Sydney)
Result
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Doing CPU performance benchmark Threads started! Done. Maximum prime number checked in CPU test: 20000 Test execution summary: total time: 39.1700s total number of events: 10000 total time taken by event execution: 39.1586 per-request statistics: min: 2.90ms avg: 3.92ms max: 20.44ms approx. 95 percentile: 7.43ms Threads fairness: events (avg/stddev): 10000.0000/0.00 execution time (avg/stddev): 39.1586/0.00 |
39.15 seconds
CPU Benchmark (Digital Ocean/London)
1 | sysbench --test=cpu --cpu-max-prime=20000 run |
Result
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Doing CPU performance benchmark Threads started! Done. Maximum prime number checked in CPU test: 20000 Test execution summary: total time: 33.4382s total number of events: 10000 total time taken by event execution: 33.4352 per-request statistics: min: 3.24ms avg: 3.34ms max: 6.45ms approx. 95 percentile: 3.45ms Threads fairness: events (avg/stddev): 10000.0000/0.00 execution time (avg/stddev): 33.4352/0.00 |
33.43 sec
CPU Benchmark (UpCloud/Singapore)
1 | sysbench --test=cpu --cpu-max-prime=20000 run |
Result
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Doing CPU performance benchmark Threads started! Done. Maximum prime number checked in CPU test: 20000 Test execution summary: total time: 23.7809s total number of events: 10000 total time taken by event execution: 23.7780 per-request statistics: min: 2.35ms avg: 2.38ms max: 6.92ms approx. 95 percentile: 2.46ms Threads fairness: events (avg/stddev): 10000.0000/0.00 execution time (avg/stddev): 23.7780/0.00 |
23.77 sec
Surprisingly, 1st place in prime generation goes to UpCloud, then Digital Ocean then Vultr. UpCloud has some good processors.
Processors:
(Lower is better)
(oops, typo in the chart should say Vultr)
Benchmark the file IO
Confirm free space
1 | df -h / |
Install Sysbench
1 | sudo apt-get install sysbench |
I had 10GB free on all servers (Vultr, Digitial Ocean and UpCloud) so I created a 10GB test file.
1 2 3 4 5 | sysbench --test=fileio --file-total-size=10G prepare sysbench 0.4.12: multi-threaded system evaluation benchmark 128 files, 81920Kb each, 10240Mb total Creating files for the test... |
Now I can run the benchmark and use the pre-created text file.
1 | sysbench --test=fileio --file-total-size=10G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run |
SysBench description from the Ubuntu manpage.
“SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load. The idea of this benchmark suite is to quickly get an impression about system performance without setting up complex database benchmarks or even without installing a database at all.”
SysBench Results (Vultr/Sydney)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Initializing random number generator from timer. Extra file open flags: 0 128 files, 80Mb each 10Gb total file size Block size 16Kb Number of random requests for random IO: 0 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Time limit exceeded, exiting... Done. Operations performed: 385920 Read, 257280 Write, 823266 Other = 1466466 Total Read 5.8887Gb Written 3.9258Gb Total transferred 9.8145Gb (33.5Mb/sec) 2143.98 Requests/sec executed Test execution summary: total time: 300.0026s total number of events: 643200 total time taken by event execution: 182.4249 per-request statistics: min: 0.01ms avg: 0.28ms max: 18.12ms approx. 95 percentile: 0.55ms Threads fairness: events (avg/stddev): 643200.0000/0.00 execution time (avg/stddev): 182.4249/0.00 |
SysBench Results (Digital Ocean/London)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Initializing random number generator from timer. Extra file open flags: 0 128 files, 80Mb each 10Gb total file size Block size 16Kb Number of random requests for random IO: 0 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Time limit exceeded, exiting... Done. Operations performed: 944280 Read, 629520 Write, 2014432 Other = 3588232 Total Read 14.409Gb Written 9.6057Gb Total transferred 24.014Gb (81.968Mb/sec) 5245.96 Requests/sec executed Test execution summary: total time: 300.0024s total number of events: 1573800 total time taken by event execution: 160.5558 per-request statistics: min: 0.00ms avg: 0.10ms max: 18.62ms approx. 95 percentile: 0.34ms Threads fairness: events (avg/stddev): 1573800.0000/0.00 execution time (avg/stddev): 160.5558/0.00 |
SysBench Results (UpCloud/Singapore)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | sysbench 0.4.12: multi-threaded system evaluation benchmark Running the test with following options: Number of threads: 1 Initializing random number generator from timer. Extra file open flags: 0 128 files, 80Mb each 10Gb total file size Block size 16Kb Number of random requests for random IO: 0 Read/Write ratio for combined random IO test: 1.50 Periodic FSYNC enabled, calling fsync() each 100 requests. Calling fsync() at the end of test, Enabled. Using synchronous I/O mode Doing random r/w test Threads started! Time limit exceeded, exiting... Done. Operations performed: 994320 Read, 662880 Write, 2121090 Other = 3778290 Total Read 15.172Gb Written 10.115Gb Total transferred 25.287Gb (86.312Mb/sec) 5523.97 Requests/sec executed Test execution summary: total time: 300.0016s total number of events: 1657200 total time taken by event execution: 107.4434 per-request statistics: min: 0.00ms avg: 0.06ms max: 15.43ms approx. 95 percentile: 0.13ms Threads fairness: events (avg/stddev): 1657200.0000/0.00 execution time (avg/stddev): 107.4434/0.00 |
Comparison
sysbench fileio results (text)
Read
Write
Other
Total Read Gb
Total Written Gb
Total Transferred Gb
Now I can remove test file io benchmark file
1 2 3 4 | sysbench --test=fileio --file-total-size=2=10G cleanup sysbench 0.4.12: multi-threaded system evaluation benchmark Removing test files... |
Confirm the test file has been deleted