Here is how I set NGINX to the development branch to get more frequent updates and features over the stable branch
Updating NGINX to the development branch (on Ubuntu) to get more frequent updates and features over the stable branch
I have a number of guides on moving away from CPanel, Setting up VM’s on UpCloud, AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. View all recent posts here https://fearby.com/all/
Now on with the post
Backup your Nginx and Server before making any changes. The Nginx development branch is quite stable but anything can happen. If your site is mission critical then stay on the stable branch.
By default, you will most likely get the stable branch of Nginx when instaling and updating Nginx. I have been running the stable version for the last few years but was made aware of a DDoS vulnerability in Nginx.
Here is a good write-up on development merges into the stable branch.
Widely-used #Nginx server releases versions 1.15.6 and 1.14.1 to patch two HTTP/2 implementation vulnerabilities that might cause excessive memory consumption (CVE-2018-16843) & CPU usage (CVE-2018-16844), allowing a remote attacker to perform #DoS attackhttps://t.co/1Z3JoghoBr pic.twitter.com/qQ3pOFD1Lk
— The Hacker News (@TheHackersNews) November 9, 2018
I was aware recently of a DDoS bug affecting Nginx and the recommendation was to update ot Nginx 1.15.6 development branch (or 1.14.1 stable branch).
A few days ago no 1.14.1 update was available but a 1.15.6 was, should I switch to the development branch to get updates earlier?
Reminder to update your #nginx installations to the 1.14.1 stable or the 1.15.6 mainline versions for critical security patches released this week. #NGINXPlus customers, see instructions for updating based on the patch released 10/30 https://t.co/KitsOWIJkb
— NGINX, Inc. (@nginx) November 8, 2018
Recent Nginx Changes
Here are the recent changes to Nginx: http://nginx.org/en/CHANGES
Changes with nginx 1.15.6 06 Nov 2018 *) Security: when using HTTP/2 a client might cause excessive memory consumption (CVE-2018-16843) and CPU usage (CVE-2018-16844). *) Security: processing of a specially crafted mp4 file with the ngx_http_mp4_module might result in worker process memory disclosure (CVE-2018-16845). *) Feature: the "proxy_socket_keepalive", "fastcgi_socket_keepalive", "grpc_socket_keepalive", "memcached_socket_keepalive", "scgi_socket_keepalive", and "uwsgi_socket_keepalive" directives. *) Bugfix: if nginx was built with OpenSSL 1.1.0 and used with OpenSSL 1.1.1, the TLS 1.3 protocol was always enabled. *) Bugfix: working with gRPC backends might result in excessive memory consumption. Changes with nginx 1.15.5 02 Oct 2018 *) Bugfix: a segmentation fault might occur in a worker process when using OpenSSL 1.1.0h or newer; the bug had appeared in 1.15.4. *) Bugfix: of minor potential bugs. Changes with nginx 1.15.4 25 Sep 2018 *) Feature: now the "ssl_early_data" directive can be used with OpenSSL. *) Bugfix: in the ngx_http_uwsgi_module. Thanks to Chris Caputo. *) Bugfix: connections with some gRPC backends might not be cached when using the "keepalive" directive. *) Bugfix: a socket leak might occur when using the "error_page" directive to redirect early request processing errors, notably errors with code 400. *) Bugfix: the "return" directive did not change the response code when returning errors if the request was redirected by the "error_page" directive. *) Bugfix: standard error pages and responses of the ngx_http_autoindex_module module used the "bgcolor" attribute, and might be displayed incorrectly when using custom color settings in browsers. Thanks to Nova DasSarma. *) Change: the logging level of the "no suitable key share" and "no suitable signature algorithm" SSL errors has been lowered from "crit" to "info". Changes with nginx 1.15.3 28 Aug 2018 *) Feature: now TLSv1.3 can be used with BoringSSL. *) Feature: the "ssl_early_data" directive, currently available with BoringSSL. *) Feature: the "keepalive_timeout" and "keepalive_requests" directives in the "upstream" block. *) Bugfix: the ngx_http_dav_module did not truncate destination file when copying a file over an existing one with the COPY method. *) Bugfix: the ngx_http_dav_module used zero access rights on the destination file and did not preserve file modification time when moving a file between different file systems with the MOVE method. *) Bugfix: the ngx_http_dav_module used default access rights when copying a file with the COPY method. *) Workaround: some clients might not work when using HTTP/2; the bug had appeared in 1.13.5. *) Bugfix: nginx could not be built with LibreSSL 2.8.0. Changes with nginx 1.15.2 24 Jul 2018 *) Feature: the $ssl_preread_protocol variable in the ngx_stream_ssl_preread_module. *) Feature: now when using the "reset_timedout_connection" directive nginx will reset connections being closed with the 444 code. *) Change: a logging level of the "http request", "https proxy request", "unsupported protocol", and "version too low" SSL errors has been lowered from "crit" to "info". *) Bugfix: DNS requests were not resent if initial sending of a request failed. *) Bugfix: the "reuseport" parameter of the "listen" directive was ignored if the number of worker processes was specified after the "listen" directive. *) Bugfix: when using OpenSSL 1.1.0 or newer it was not possible to switch off "ssl_prefer_server_ciphers" in a virtual server if it was switched on in the default server. *) Bugfix: SSL session reuse with upstream servers did not work with the TLS 1.3 protocol. Changes with nginx 1.15.1 03 Jul 2018 *) Feature: the "random" directive inside the "upstream" block. *) Feature: improved performance when using the "hash" and "ip_hash" directives with the "zone" directive. *) Feature: the "reuseport" parameter of the "listen" directive now uses SO_REUSEPORT_LB on FreeBSD 12. *) Bugfix: HTTP/2 server push did not work if SSL was terminated by a proxy server in front of nginx. *) Bugfix: the "tcp_nopush" directive was always used on backend connections. *) Bugfix: sending a disk-buffered request body to a gRPC backend might fail. Changes with nginx 1.15.0 05 Jun 2018 *) Change: the "ssl" directive is deprecated; the "ssl" parameter of the "listen" directive should be used instead. *) Change: now nginx detects missing SSL certificates during configuration testing when using the "ssl" parameter of the "listen" directive. *) Feature: now the stream module can handle multiple incoming UDP datagrams from a client within a single session. *) Bugfix: it was possible to specify an incorrect response code in the "proxy_cache_valid" directive. *) Bugfix: nginx could not be built by gcc 8.1. *) Bugfix: logging to syslog stopped on local IP address changes. *) Bugfix: nginx could not be built by clang with CUDA SDK installed; the bug had appeared in 1.13.8. *) Bugfix: "getsockopt(TCP_FASTOPEN) ... failed" messages might appear in logs during binary upgrade when using unix domain listen sockets on FreeBSD. *) Bugfix: nginx could not be built on Fedora 28 Linux. *) Bugfix: request processing rate might exceed configured rate when using the "limit_req" directive. *) Bugfix: in handling of client addresses when using unix domain listen sockets to work with datagrams on Linux. *) Bugfix: in memory allocation error handling.
Development branch changes are made every few weeks and stable branch changes are made less often.
Normally you update Nginx bu running an update and upgrade
apt-get update && apt-get upgrade
Restart Nginx for good measure
Checking NGINX Version
nginx -v nginx version: nginx/1.14.1
Changing your repository to the development branch
I changed ot the development branch by running
sudo add-apt-repository ppa:nginx/development
Update and upgrade Nginx
apt-get update && apt-get upgrade
Restart Nginx for good measure
Checking NGINX Version
nginx -v nginx version: nginx/1.16.6
Removing the stable Nginx repository
Run this command to remove the stable branch of Nginx
sudo add-apt-repository -r ppa:nginx/stable
Check to see if the development branch is listed
grep -r --include '*.list' '^deb ' /etc/apt/sources.list* |grep nginx /etc/apt/sources.list.d/nginx-ubuntu-development-bionic.list:deb http://ppa.launchpad.net/nginx/development/ubuntu bionic main
Good luck and I hope this guide helps someone
Ask a question or recommend an article
v1.0 Initial post
Below are the manual Task’s I perform frequently on all self-managed Ubuntu serves I own (along with automated backups and update tasks).
I also perform manual backups to ensure files are backed up.
Manual Backup Files.
First I create the following folder structure on my OSX desktop (for each server I need to backup).
My Server 01
My Server 02
etc (for each server)
I then use Forklift 3 (not a paid endorsement, I just love it) to manually backup files on the server. I manually copy files that are available via existing SFTP connections in Forklift (e.g WWW, MySQL, NGINX, MongoDB etc).
I simply drag and drop important file system files in Forklift (from the remote SFTP instance to a local folder).
SFTP copy progress can be viewed in Forklift.
fyi: SFTP is not the fastest transfer protocol. It appears only 39MB (4055 items) has been downloaded in 8 hours over ADSL (on a server that is over 400ms away).
FYI: Slow servers (ping) do not like like SFTP.
ping myserver01.com PING myserver01 (ip_removed): 56 data bytes 64 bytes from 45.x.x.x: icmp_seq=0 ttl=53 time=450.338 ms 64 bytes from 45.x.x.x: icmp_seq=1 ttl=52 time=423.412 ms 64 bytes from 45.x.x.x: icmp_seq=2 ttl=52 time=458.129 ms 64 bytes from 45.x.x.x: icmp_seq=3 ttl=53 time=462.419 ms
TIP: Consider zipping smaller files first (into one larger file) or using RSync instead.
Pre Zipping up files before backing up.
I used this command to pre-compress entire folders before downloading them over SFTP.
cd / && zip -r -9 /www.zip /www && zip -r -9 /nginx.zip /etc/nginx/ && ls -al
I created a bash script to manually prepare files to backup on each server.
#!/bin/bash sudo rm /www.zip -R sudo rm /nginx.zip -R sudo zip -r -9 /www.zip /www sudo zip -r -9 /nginx.zip /etc/nginx/
Don’t forget to make the script file executable
chmod +X _ManualBackup.sh
Now I call the script to compress files before backup
sudo bash _ManualBackup.sh
TIP: Add other things to back up to the script (e.g MongoDB and MySQL).
Below is my script to backup WWW, NGINX, MySQL and MongoDB and zip them before copying over SFTP.
#!/bin/bash # Backup WWW Task echo "Backing up WWW" sudo rm /www.zip -R sudo zip -r -9 /www.zip /www echo "Finished Backing up WWW" # Backup NGINX Task echo "Backing up NGINX" sudo rm /nginx.zip -R sudo zip -r -9 /nginx.zip /etc/nginx/ echo "Finished backing up NGINX" #Backup MySQL Task echo "Backing up MYSQL Databases" sudo rm /*.sql USER="***********" PASSWORD="*********************************" databases=`mysql -u $USER -p$PASSWORD -e "SHOW DATABASES;" | tr -d "| " | grep -v Database` for db in $databases; do if [[ "$db" != "information_schema" ]] && [[ "$db" != "performance_schema" ]] && [[ "$db" != "mysql" ]] && [[ "$db" != _* ]] ; then echo "Dumping database: $db" mysqldump -u $USER -p$PASSWORD --databases $db > /$db.sql fi done sudo zip -r -9 /dbs.zip /*.sql rm /*.sql -R echo "Finsihed Backing up MYSQL Databases" # Backup MongoDB Task echo "Backing up MongoDB" rm /mondodb.zip -R sudo zip -r -9 /mongodb.zip /mongodb* echo "Finsihed Backing up MongoDB" # Done ls /*.zip -al echo "Done"
Thanks to this thread for MySQL export to separate file help.
Note: The warning “Using a password on the command line interface can be insecure.” will be shown when exporting a database from the CLI.
Now I can download the 4 outputted files manually from each server.
cd / ls -al -rw-r--r-- 1 username username 1615478 Dec 30 13:47 /dbs.zip -rw-r--r-- 1 username username 753094 Dec 30 13:48 /mongodb.zip -rw-r--r-- 1 username username 42222 Dec 30 13:47 /nginx.zip -rw-r--r-- 1 username username 239327652 Dec 30 13:47 /www.zip
Taking snapshots in Vultr is a great way to backup too.
Manually Updating the Server
I also connect to the server via SSH and check for package updates
/usr/lib/update-notifier/apt-check --human-readable 9 packages can be updated. 3 updates are security updates.
Now I can now manually update packages
sudo apt-get update && sudo apt-get upgrade
Quick Reboot to ensure packages are updated.
sudo shutdown -r now
More to come (on viewing logs (errors, firewall, stats), backing up php etc).
Hope this helps someone.
Donate and make this blog better
Ask a question or recommend an article
v1.0 Initial Version