Code, InfoSec and Server Stuff

Views are my own and not my employer's

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

"If you're not still learning, you're already dying."
- Ryan Holiday - Ego is the Enemy

Follow me on Twitter: @FearbySoftware

View All Posts.

cpu

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

by

Before UpCloud, when I had 512MB ram on Vultr I had multiple NGINX crashed a day so I used a bash script and scheduled a cron job to clear memory cache when memory fell below 100MB (view the script here).  To further increase the speed of the WordPress I have configured the OS to use memory over the disk.  About once a day free memory falls below 100MB (this is not a problem as my script clears cached items automatically).

Read the full article here: https://fearby.com/article/upgrading-an-ubuntu-server-on-upcloud-to-add-more-cpu-memory-and-disk-space/

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

by

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

Advertisement:



If you have not read my previous posts I have now moved my blog from Vultr to the awesome UpCloud host (signup using this link to get $25 free credit).

Recently I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

Spoiler: UpCloud performance is great.

Upcloud Site Speed in GTMetrix

Why Upgrade

I have 1 CPU, 1 GB memory and 50GB storage and it is running well?  I have PHP child workers set up and have set up the preferred use of memory over swap file usage.

View of htop querying processes on Ubuntu

Before UpCloud, when I had 512MB ram on Vultr I had multiple NGINX crashed a day so I used a bash script and scheduled a cron job to clear memory cache when memory fell below 100MB (view the script here).  To further increase the speed of the WordPress I have configured the OS to use memory over the disk.  About once a day free memory falls below 100MB (this is not a problem as my script clears cached items automatically).

Graph of memory falling below 100MB every day

I’d like to add more memory as I am working on some things (watch this space) and I will use the extra memory. I’d prefer the server is set up now for the expected workload.

How to Upgrade

This is how I upgraded from 1xCPU/2GB Memory/50GB Storage/2TB Transferred Data to a 2 CPU/4GB Mmeory/80GB Storage/4TB Transferred Data server.

UpCloud Pricing: https://www.upcloud.com/pricing/

Pricing table form https://www.upcloud.com/pricing/

Upgrade an UpCloud VM

I shut down my existing VM. Read this guide to setup a VM.

Login to the UpCloud dashboard, select your server (confirm the server has shut down) and click General Settings, choose the upgrade and click Update.

Upgrade the Server, shut it down the server and choose upgrade

I confirmed the upgrade options (2x CPU, 4096 MB Memory).

Confirm Upgrade Options

Click Update

Upgrade Applied

After 10 seconds you can start your server from the UpCloud server.

I confirmed the CPU upgrade was visible in the VM

Software Tweaks Post Upgrade.

I added these settings to the top of /etc/nginx/nginx.conf to ensure the extra CPU was used.

I increased PHP FPM ( /etc/php/7.2/fpm/php.ini ) to increase the memory usage and child workers. I doubled child workers and max memeory limit.

I restarted NGINX and PHP

I tweaked WordPress max memory limits

MySQL Tweaks: I logged into MySQL

I ran “SHOW GLOBAL STATUS” to view stats

Read more on SHOW GLOBAL STATUS here. Read more on the values here.

I can see NO major errors here (possibly due to UpClouds awesome disk IO) so I won’t be making memory tweaks in MySQL. Sign Up using this link and get $25 credit free on UpCloud and see for yourself how fast they are.

Configure Ubuntu System Memory Usage

Edit /etc/sysctl.conf

Add the following to allow things to sit in ram longer

Snip from: https://www.kernel.org/doc/Documentation/sysctl/vm.txt

Read these pages here and here regarding setting MySQL memory.

Reboot

Resize the disk

The 2x CPU, 4GB memory plan comes with 80GB storage allowance.  My disk at present is 50GB and I will update the size soon following this guide soon.

Upgrade disk from 50gb to 80GB soon

Quick Benchmark

I used loader.io to load 500 users to access my site in 1 minute.

HTOP showing 2x busy CPU's running at 60%

The benchmark worked with no errors.

Loader.io Success with 500 concurrent users

This benchmark was performed with no Cloudflare caching. I should get Cloudflare caching working again to lower the average response time. I loaded my website manually in Google Chrome while loader.io threw 500 users at my site and it loaded very fast.

Conclusion

After a few days, I checked my memory logs and there were no low memory triggers (just normal internal memory management triggers). Ubuntu was happier.

No Low memory low triggers

This graph was taken before I set “vm.vfs_cache_pressure” so I will update this graph in a few days.

I hope this guide helps someone.

Please consider using my referral code and get $25 credit for free.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

1.0 Initial Post

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 4 of 4

by

How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 4 of 4

Advertisement:



Read Part 1, Part 2, Part 3 or Part 4

I ran the MySQL benchmark preparation command again (no problem this time).

Test table and records created

Test Records Created

Now I can benchmark MySQL on my main server.

RAW Output

Results

queries performed (in 60 seconds):

  • read: 336210
  • other: 48030
  • total: 384240

I decided to add an index to see if I can speed this query up (read the MySQL index page here). I added an index (in Adminer) on the columns “Id” and “pad” for the sbtest table in the test database

I restarted the MySQL process

I ran the same benchmark again.

Raw Output

Results

The quick index added 20% extra throughput on queries 🙂

Mysql before and after an index

Don’t forget to delete your test database

Viewing MySQL Index Usage (on the “test” database)

Query to show Index stats for a table ‘test’

I can see the MySQL PRIMARY index is getting used 🙂

Index Summary

Read more in viewable query stats (columns) here.

Other System Information Tools

Show processor information

Output

Memory Information

You can assign 512MB, 1GB, 2GB or more memory to a server on Vultr, Read my guide here on upgrading resources for Vultr VM’s here.

Only upgrade your server’s memory when server processes demand it, there is no need to pay for extra idle memory. Read my older guides on upgrading Digital Ocean and AWS servers.

I use the htop utility to monitor memory and processes. The memory usage will depend on how you have configured your server to use connection pools in code, MySQL or services.  Also what memory demands do you get in pean bandwidth times?

HTOP

You can check your server memory details on Ubuntu with this command

Output

Use Memory or Disk (Swap)

You can configure the use of Memory over Disk by configuring your/etc/sysctl.conf file (setting value “vm.swappiness”)

You can check your swap file settings by running the following command

Or By running

Set a new swap file value by editing /etc/sysctl.conf

Set the following to use more ram over the swap disk.

Read about swappiness values here: https://en.wikipedia.org/wiki/Swappiness

Service Performance

Performance (and allocated resources) depends on the demands of your operating system and installed software

What operating system do you have?

View NGINX Status, how much memory does it use?

PHP (and Child Worker) status how much memory does it use and how many child workers do you have? Read my add PHP child workers post here (and update to PHP 7.2 here)

MySQL Status

Shared VM Hosts

One of the biggest impacts (after server latency) for your server is not the disk performance but the number of hosts/websites on the server who are also using the disk and server resources.

Reverse IP Lookup

I have 80 other web servers on my server (based on a reverse lookup).

I may move to a dedicated box when I can afford it.

Security

Above all else ensure that security is number 1 priority and make performance second priority.

Scan your site with Zap, Qualys and Kali Linux. Performance means nothing if you are hacked.

website-report

Simulated concurrent users

You can use Siege to test the maximum concurrent users accessing your site before the server starts to drop connections.

FYI: If you use Cloudflare (you should) this may not work as it will block connections.

Install Siege

Test  your server with 10 concurrent serves for 1 minute

Results

Keep upping the connections (from 10 above) to a limit where connections start dropping.

I tried 25 then 50 concurrent users hitting a server on Digital Ocean and it did not fail.

Conclusion

  • Choose a server near your customers
  • Change hosts if one is faster and cheaper
  • Measure or benchmark your server (and compare over time).
  • Use Cloudflare

Create your own server today

  • Create your own server on Vultr here.
  • Create your own server on Digital Ocean here.
  • Create your own server on UpCloud here.

And remember you can install the Runcloud server management dashboard here.

I hope this guide helps someone.

 

< Previous

Read Part 1, Part 2, Part 3 or Part 4

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 3 of 4

by

How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 3 of 4

Advertisement:



Read Part 1, Part 2, Part 3 or Part 4

I used these commands to generate bonnie++ reports from the data in part 2

Image of results here

Bonnie Results

Network Performace

IMHO Network Latency is the biggest impact on server performance, Read my old post on scalability on a budget here. I am in Australia an having a server in Singapore was too far away and latency was terrible.

Here is a non-scientific example of pinging a Vultr, Digital Ocean and UpCloud server in three different locations (and Google).

Ping Test

Test Ping Results

Latency matters, run a https://www.webpagetest.org/ scan over your site to see why.

Adding https added almost 0.7 seconds to https communications in the past on Digital Ocean (a few thousand kilometres away). The longer the latency the longer HTTPS handshakes take.

SSL

Deploying a server to Singapore (in my experience) is bad if your visitors are in Australia. But deploying to other regions may be lower in cost though. It’s a trade off.

Server Location

Deploy servers as close as you can to your customers is the best tip for performance.

Deploy serves close to your customers

Also, consider setting up Image Optimization and Image CDN plugins (guide here) in WordPress and using Cloudflare (guide here)

Benchmarking with SysBench

Install CPU Benchmark

CPU Benchmark (Vultr/Sydney)

Result

39.15 seconds

CPU Benchmark (Digital Ocean/London)

Result

33.43 sec

CPU Benchmark (UpCloud/Singapore)

Result

23.77 sec

Surprisingly, 1st place in prime generation goes to UpCloud, then Digital Ocean then Vultr.  UpCloud has some good processors.

Processors:

  • UpCLoud (Singapore): Intel(R) Xeon(R) CPU E5-2687W v4 @ 3.00GHz
  • Digital Ocean (London): Intel(R) Xeon(R) CPU E5-2630L v2 @ 2.40GHz
  • Vultr (Sydney): Virtual CPU a7769a6388d5 (Masked/Hidden CPU @ 2.40GHz)

(Lower is better)

prime benchmark results

(oops, typo in the chart should say Vultr)

Benchmark the file IO

Confirm free space

Install Sysbench

I had 10GB free on all servers (Vultr, Digitial Ocean and UpCloud) so I created a 10GB test file.

Now I can run the benchmark and use the pre-created text file.

SysBench description from the Ubuntu manpage.

“SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load. The idea of this benchmark suite is to quickly get an impression about system performance without setting up complex database benchmarks or even without installing a database at all.”

SysBench Results (Vultr/Sydney)

SysBench Results (Digital Ocean/London)

SysBench Results (UpCloud/Singapore)

Comparison

Sysbench Results table

sysbench fileio results (text)

Read

  • Vultr (Sydney): 385,920
  • Digital Ocean (London): 944,280
  • UpCloud (Singapore): 944,320

Write

  • Vultr (Sydney): 823,266
  • Digital Ocean (London): 629,520
  • UpCloud (Singapore): 662,880

Other

  • Vultr (Sydney): 1,466,466
  • Digital Ocean (London): 3,588,232
  • UpCloud (Singapore): 2,121,090

Total Read Gb

  • Vultr (Sydney): 5.8887 Gb
  • Digital Ocean (London): 14.409 Gb
  • UpCloud (Singapore): 15.172 Gb

Total Written Gb

  • Vultr (Sydney): 3.9258 Gb
  • Digital Ocean (London): 9.6057 Gb
  • UpCloud (Singapore): 10.115 Gb

Total Transferred Gb

  • Vultr (Sydney): 9.8145 Gb
  • Digital Ocean (London): 24.014 Gb
  • UpCloud (Singapore): 25.287 Gb

Now I can remove test file io benchmark file

Confirm the test file has been deleted

Bonus: Benchmark MySQL (on my main server (Vultr) not on Digital Ocean and UpCLoud)

I tried to run a command

To fix the error I created a test table with Adminer (guide here).

Create Test Table

< Previous – Next >

Read Part 1, Part 2, Part 3 or Part 4

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4

by

How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4

Advertisement:



Read Part 1, Part 2, Part 3 or Part 4

Measure Disk Performance with Bonnie++

Installing Bonnie++ on Ubuntu

Read this. post on using Bonnie++

Benchmark disk IO with DD and Bonnie++

Starting Bonnie++

Bonnie++ Readme.

Disk io with bonnie++ on Vultr/Sydney

Disk io with bonnie++ on Digital Ocean/London

Disk io with bonnie++ on UpCloud/Singapore

Now read this site on how to make sense of this data

 

< Previous – Next >

Read Part 1, Part 2, Part 3 or Part 4

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4

by

How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4. Update: I moved my domain to UpCloud.

Advertisement:



Update (June 2018): I moved my domain to UpCloud (they are that awesome). Use this link to signup and get $25 free credit. Read the steps I took to move my domain to UpCloud here.

Upcloud Site Speed in GTMetrix

Comparing Digital Ocean/Vultr and UpCloud Disk IO

I have a number of guides on moving away from CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean (all in the search of extra performance) but how do you know when a server performance is ok apart from running GT Metrix and other external site benchmarking tools.

This post is split up as it was too long.

Read Part 1, Part 2, Part 3 or Part 4

Spoiler: It all depends on where your server is located and what you do with it (Tweaks will improve the performance).

P.S This is NOT a paid endorsement or conclusive test (just a quick benchmark/review).

What does your server do?

You need to know what your server does 24/7 and what resources the services need.

I use htop to view real-time and historical usage data for each process.

htop

Tweaking Advice

A friend gave me good advice re-tweaking a cheap host to get good performance

I am not a fan of just throwing more money at a host and expecting better performance. Host have unique features and cons., there is no shortage of hosts or host cons.

How can you run synthetic benchmarks to determine comparable performance metrics?

WARNING: Comparing synthetic benchmarks can be far removed from real-world speeds. Benchmark results below were from 3 different servers I have on 3 different hosts in three different locations (the only thing the same was the use of Ubuntu 16.04 $5/m servers). These results are not scientific and should not be used to compare host providers. Benchmark runs were one-off (not averages over multiple timezones/days).

Disk Performance

Speaking of disk performance I noticed this the other day on the RunCloud blog. Faster than SSD (UpCloud)?

UpCloud Faster-than-SSD Cloud Hosting Server (Promo Code Inside)

Runcloud is a server management console that can interface with your domains (read my old review here).  I don’t use Runcloud but it is great for those who need a GUI to help manage VM via a dashboard. However, I prefer to know what is going on under the hood. I have investigated webmin in the past though.

Let’s do a quick IO benchmark test between UpCloud, Digital Ocean and Vultr on similarly low end $5/m servers,

Good advice on command line benchmarking tools from a friend.


Installing iozone to test disk performance

I searched for a post on using iozone (Thanks thegeekstuff).  I will be reviewing the “Writer report” and “Reader report”. Read more about iozone here.

View the iozone page for how to break down results.

iozone results breakdown

(image snip from http://www.iozone.org/)

Install iozone on Ubuntu

Run an iozone disk test and output the results to a spreadsheet.

Now let’s run a Read/Write test on Vultr/Digital Ocean and UpCloud. Multiple runs were not performed, this is not a scientific test (just a simple benchmark test (as is, ignoring sever load and local infrastructure/timezone load)).

iozone Benchmark results for Vultr “Read” (Sydney)

 “4” “8” “16” “32” “64” “128” “256” “512” “1024” “2048” “4096” “8192” “16384”
6421337303363612427406245647866421025
1282248149353656641359587082197413595811720614
2561884399269916138790453667079597167856870205687020
51231404883736016368473342625234610256263881650671425684095
1024161780819392073411938399976240487784614246308368058850836609617
2048192651025696784423683499761839370754596052896324354252449715854707314
409617016832151300420992050017004751325486984553892463647681492852162070354347346
8192206342423293463203763293728032214853232699362643136507063789200411060337150454350542
163841738553277836233976133679205369344231715013524291339358630040243552531345657426938452488861
327680000295289435371533574875376815547196133890280339499527352222542914
655360000405748936107893619967380007832753273591212360718817704262826659
1310720000355227018907425275167372733935276071753893323473623411111378601
2621440000379858613020211491429371282532288163757963371551025924852481061
5242880000275875624879233705741180732821183093675988319636733943302396842

iozone Benchmark results for Digital Ocean “Read” (London)

4 “8” “16” “32” “64” “128” “256”  “512”  “1024”  “2048” “4096”  “8192”  “16384”
644564786710039790061791040217812902017
12847174347082197854812497958961056714010779307
256484091170731328271916986843310148242106515981E+07
51247426166909408814039993042929638369100440891E+0710044089
102442490535917516620834375375999300377104549847E+0671131619946527
204838854316967792660354968456291040188398080369E+06790383693084977817519
40962506983595323162636116953144777437962250286E+068081580768397280815808240513
81923665114485046354793176141364627712061086086E+0665699835732541716603366334025479317
163843673501482858454161826187150661476162988726E+06643031059840336402750604615947918833405527
327680000469254261409296E+06629564252312246545707578110845134753702577
655360000631543058301316E+06644469562191256473838533859542481183679324
1310720000613000264614966E+06595806859834236387547613807839948883602079
2621440000645674663237276E+06650414663901766486151643396339551653654188
5242880000166733763814566E+06644570864487146421071598120041551853770740

iozone Benchmark results for UpCloud “Read” (Singapore)

 “4” “8” “16” “32” “64” “128” “256” “512” “1024” “2048” “4096” “8192” “16384”
6464210256421025108215241290201715972885
128488928164061389129573107793071420079414200794
25653206713879045107583228815202102450711281227712228612
512430525051154228844453823403670919528394979754017010235583
10244339202476263058212716163794681951146745106479979818391810230845
20484204968531948458008515816563624356663780055953632685108979403678229438
409645260135556581481794854045047301864575963458102806007355691953886209456281934
8192429829550190935927357603670267813416082655585563665275466553692679206564661264437634
163844282172584955863139196635840674195866570546423097553662265585756442970452703237847773901898
327680000582546054234086504198666538563653296426343526307637186053705971
655360000690807566231166493259660973863118056483610548967440359823561526
1310720000565018057189492465429539125334959115784844536740837334903582175
2621440000681462766912506189661590678660816455799913524791941212503637601
5242880000640476463092635673979575160962882456305103597868039119843767116

iozone Benchmark results for Vultr “Write” (Sydney)

 “4” “8” “16” “32” “64” “128” “256” “512” “1024” “2048” “4096” “8192” “16384”
64289322532815507625429630566551
128398921465304434078417212669577821147
256530031613985820398474937891956815414370025
512387576754083709019819085702295609421924123496091
102429723344852271608992348885407381734012031371072453601636
204840869763465569538313581345496571295458821154797520964207258493
409623615043380412157741245025820832809958137133991426992108310046821481431
819261111366667780628671521977982582429487594787009110463787911921023592453248
16384435454706149718313845499893495888068812778842885820591941120839610862672406590
327680000465196786067938881627294890917968147872369871329842843
655360000515057790172937568915601897235867197907562852002743856
13107200005010914804928131478708868802398053336846301117578633185
2621440000387126323185323656473258405744369599422554468992453563
5242880000325588380450392965451608303255355148386250432054416512

iozone Benchmark results for Digital Ocean “Write” (London)

 “4” “8” “16” “32” “64” “128” “256” “512” “1024” “2048” “4096” “8192” “16384”
64831569566551127944713639611392258
1286524881319723142102399089116631391561553
2561185399115232315343421598292182669517075891514860
51211665991296159139918916209801620980136192015897791672748
1024107919013212001584972191756215926121701108171812014629601643814
20481210394147017216217191550584179637816437531713598175958116491171488257
40969165131287575157471814065941742237173414816524181583280159934616610451533532
8192110974513187481178567154420115023401371492146674714995211479759156487812912921347609
163841106205128208413740371503649142939814614071496119157813215472891333431120337111988151501316
327680000127091414065891513114146822615583031552038151633614432801440360
655360000131932213279841311504141195512669881359645138644613470921368295
1310720000110065812293261227197131863112655521233306122774712378961233502
2621440000116716010640781155828118518510861521193673108087210626111141960
524288000097783511248161052757121918311289721140177109195411416351132063

iozone Benchmark results for UpCloud “Write” (Singapore)

 “8” “16” “32” “64” “128” “256” “512” “1024” “2048” “4096” “8192” “16384”
6411432231255511156243614525281279447
128145176414061361543594150465918525201749872
2561642294182980819708711855098180216719529472000242
51215374241854787180187322947961983258212452618957211417662
1024143413815534421609925193135920983752044438187241917683451892218
20481562145190177118172811848169196709712962402267786208149719157682007554
409616253721966378192474113420921950306207817519148731459656199515221028491326855
8192144406218083301956503192439721273002042328213563019864782062557206131913370161812049
163841667066182024818984952051339201253021110802119806149121720608751974254193478918158231921911
327680000205750614545372075621207089918697952052896189234718553821873440
655360000206712720776732088994217980920874712099108190472316425051832204
1310720000123466318249591304340177551412874811560379163199210856091675467
2621440000685774808487823824662524681762548308814946645663732176
5242880000547296517384503422521173538714518429528950529593512944

Here is my quick unscientific take on a one pass benchmark results above.

Vultr (Read) Vultr (Write) Digital Ocean (Write) UpCloud (Read) UpCloud (Write)

These results need some decoding.

Next >>

Read Part 1, Part 2, Part 3 or Part 4

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Update (June 2018): I moved my domain to UpCloud (they are that awesome). Use this link to signup and get $25 free credit. Read the steps I took to move my domain to UpCloud here.

Upcloud Site Speed in GTMetrix

Revision History

v1.2 added the fact that I Moved to UpCloud.

v1.1 Re ran iozone -a -b iozone.xls on all servers.

v1.0 Initial post

Upgrading the RAM, CPU and Memory on a Vultr Ubuntu VM in the cloud

by

Upgrading the RAM, CPU and Memory on a Vultr Ubuntu VM in the cloud is quite simple.

Advertisement:



I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line.  I prefer Vultr as they are located in the country (Australia) and are easy to use.

First, you need to shut down the server from within the VM (SSH), I used the command.

Once the VM is shut down (wait a few minutes) you can turn off the VM in the Vultr GUI.

Shutdown

You can then go to Settings, Change Plan and review upgrade options.

Upgrade Options

Snapshot

Don’t forget to take a final snapshot.

Snapshot reminder

Goto the Snapshots page (read this guide to restore a snapshot) and click Take Snapshot.

Take Snapshop

You can see snapshot progress on the main screen.

Snapshot Progress

It may take a while for your snapshot to change from Pending to Processing.

Processing

Upgrade

When the snapshot is done it will auto boot and allow you to upgrade.

Manage

Choose the Upgrade specifications (Settings, Change Plan)

Upgrade Specs

Click Upgrade

Upgrade

Confirm