I have moved from expensive C-Panel hosted domains to self-managed Ubuntu servers and have collected a number of Ubuntu Terminal (command line) tips and tricks
Linux
Useful Linux Terminal Commands
Below are Ubuntu Linux commands I use often to setup and maintain servers.
I recently moved my domain from a C-Panel hosted domain (and Email to Google G Suite (my guide here)) to a self-managed Digital Ocean domain (my LetsEncrypt Guide here, my Digital Ocean guide here, my AWS setup guide here, my Vultr setup guide here) and needed to transfer my WordPress site. Setup your own Digital Ocean Ubuntu server form $5 a month (get the first 2 months free by clicking here) or setup your own Vultr Ubuntu server for as low as $2.5/month by clicking here.
How to Reboot (from time to time when prompted with *** system restart required ***” messages appear).
|
1 |
sudo reboot now |
How to setup a bash file (*.sh) as Executable.
|
1 |
chmod +X filename.sh |
The file will now be executable.
Viewing your crontab (Windows Task Scheduler equiv)
|
1 |
crontab -e |
Run a file every 1 minute
|
1 |
*/1 * * * * /scripts/script1.sh |
Show service status
|
1 |
service --status-all |
View your bash history for a past command (e.g “openssl”).
|
1 2 |
grep "openssl" ~/.bash_history openssl req -new -newkey rsa:4096 -nodes -keyout fearby.key -out fearby.csr |
View last 1 line of a file.
|
1 2 |
tail -n 1 index.html </body></html> |
Ping a server
|
1 |
ping -c 2 fearby.com |
How to check system uptime (and load average)
|
1 2 |
uptime > 12:18:48 up 1 min, 1 user, load average: 0.30, 0.18, 0.17 |
The load averages at the end are the last 1, 5min and 15 minutes.
The w command is handy for showing uptime information as-well as logged in users (The w with parameter -i is handy for seeing what IP people are logged in from).
|
1 2 3 4 |
w -i > 12:22:41 up 5 min, 1 user, load average: 0.00, 0.07, 0.05 USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT root pts/0 123.123.123.123 12:18 1.00s 0.07s 0.00s w |
How to download a file
|
1 |
url -o localfile.zip http://www.serverwhereiuploadedthefile.com/remotefile.zip |
How to edit NGINX configuration
|
1 |
sudo nano /etc/nginx/nginx.conf |
and
|
1 |
sudo nano /etc/nginx/sites-available/default |
How to find a file
|
1 |
find / -name filename.ext |
also
|
1 |
locate php.ini |
Find contents in files (recursive)
|
1 |
grep -r "ahref" * |
Find files by name and run a command on each
|
1 |
find -iname "index.html" -exec md5sum {} \; |
Show differences in files
|
1 2 3 4 |
diff index.html index2.html < <body>Loading <a href="http://simon.fearby.com/blog/">http://simon.fearby.com/blog/</a></body></html> --- > <body>Loading <a href="http://www.fearby.com/blog/">http://www.fearby.com/blog/</a></body></html> |
Show contents of file (e.g urls.txt)
|
1 2 3 4 5 6 |
cat urls.txt http://www.server1.com http://www.server2.com http://www.server3.com http://www.server4.com http://www.server5.com |
Download all files mentioned in a text file.
|
1 2 3 |
cat urls.txt | xargs wget –c .. download 4 files ... cat urls.txt | xargs wget –c |
Backup MySQL
|
1 |
mysqldump --all-databases > /backup/dump-$( date '+%Y-%m-%d_%H-%M-%S' ).sql -u root -p |
Add this to crontab -e to backup at 1 am every day
|
1 |
0 1 * * * /usr/bin/mysqldump --all-databases > /backup/mysql/dump-$( date '+%Y-%m-%d_%H-%M-%S' ).sql -u root -pmysqlpassword |
/scripts/shrinkmysql.sh script to shrink SQL files
|
1 2 3 4 5 |
sudo nano /scripts/shrinkmysql.sh #!/bin/bash tar -zcf /backup/mysql-$( date '+%Y-%m-%d_%H-%M-%S' ).tgz /backup/mysql/ rm /backup/mysql/*.sql |
I had to add this to crontab
|
1 2 |
SHELL=/bin/sh PATH=/usr/local/sbin:/usr/local/bin/:/sbin:/usr/sbin:/scripts/: |
Cron job to shrink sql dumps at 2am every day
|
1 |
0 2 * * * /bin/bash /scripts/shrinkmysql.sh > /dev/null 2>&1 |
Write to a single log file from the crontab at 2am every day.
todo
Query a package (e.g. siege package)
|
1 |
sudo dpkg-query -l | grep siege * |
How to setup a free SSL certificate (see my guide here).
How to set your timezone.
|
1 |
dpkg-reconfigure tzdata |
How to restart PHP
|
1 |
sudo systemctl restart php7.0-fpm |
How to show the time on the server
|
1 |
sudo hwclock --show |
Reload and restart the NGINX configuration and web server.
|
1 2 3 |
sudo nginx -t sudo nginx -s reload sudo /etc/init.d/nginx restart |
JSON viewing program
Installing
|
1 |
sudo apt-get install jq |
Using
|
1 |
curl 'https://api.github.com/repos/stedolan/jq/commits?per_page=5' | jq . |
Below are utilities I use a lot.
ncdu file size utility
Installing
|
1 |
sudo apt-get install ncdu |
Using
|
1 |
sudo ncdu / |
pydf disk checking utility
Installing
|
1 |
sudo apt-get install pydf |
Using
|
1 |
pydf |
ntp timezone service
Installing
|
1 |
sudo apt-get install ntp |
Using
|
1 |
ntp |
Displaying startup processes
Installing
|
1 |
sudo apt-get install rcconf |
Using
|
1 |
sudo rcconf |
htop process manager.
Installing
|
1 |
sudo apt-get install htop |
Using
|
1 |
htop |
Network Benchmarking (between two servers)
I use iperf to measure total bandwidth between two servers. You will need to allow port 5001 (TCP IPV4 and TCP IPV6 in and out) in any local firewalls and hosts GUIs.
Allow port 5001 on an ufw firewall.
|
1 2 3 |
sudo ufw allow 5001 #I Set port 5001 firewall on my hosts (Digital Ocean and Vultr GUI) |
Install iperf on the target and source Ubuntu server.
|
1 |
sudo apt-get install iperf |
Run this on the listening server.
|
1 |
iperf -s |
Run this on benchmarking server (and add the IP of the listening server).
|
1 |
iperf -c 123.123.123.123 |
Results
Testing concurrent connections to a web server with siege.
Install siege
|
1 |
sudo apt-get install siege |
Benchmarking a HTTP server
|
1 2 3 |
./siege -t1m c10 'http://fearby.com' #10 Concurent users |
Benchmarking HTTPS sites
Install siege 4.0.2 ( steps here ).
Verify siege 4.0.2 is installed by running
|
1 |
siege -v |
Now can benchmark https sites
|
1 |
./siege -t1m c10 'https:/thedomain.com' |
View incoming connections on the target server
|
1 |
sudo netstat -tupn |
I always increase my history size and tell it not to store duplicates.
Viewing your typed terminal history
|
1 |
history |
Increasing your history size
|
1 2 |
HISTSIZE=10000 HISTCONTROL=ignoredups |
How to Update Ubuntu
|
1 2 |
sudo apt-get update sudo apt-get dist-upgrade |
Edit SSH authorized keys
|
1 |
sudo nano ~/.ssh/authorized_keys |
Search file and show lines where text matches
|
1 2 |
grep -i "href" index.html > <body>Loading <a href="http://simon.fearby.com/blog/">http://simon.fearby.com/blog/</a></body></html> |
more soon…
50 most useful linux commands (view here).
View other Linux command informing sites here, here and here.
Donate and make this blog better
Ask a question or recommend an article
v1.2 added search bash history command
Setting up a Raspberry Pi Zero W
fyi: Here is my guide that I created while I set up a Raspberry Pi Zero W. My previous Raspberry PI 2 Setup Guide here and I wanted to try the single core Raspberry Pi Zero W. Eventually, want to run my Raspberry Pi from batteries (my older guide here).
I plugged in a 2Amp 5v micro USB powder pack, micro HDMI cable, micro USB keyboard and expected to see a light but nothing. This guide says the Raspberry Pi has no power LED. Unfortunately, I thought the power pack was faulty and I plugged it in a few times and corrupted the installed SD-card.
Failed Setup
After reading this guide I remembered I forgot to switch the monitor to the HDMI input (oops).
Yep, I realized I corrupted the OS on the SD Card.
Old Noobs Operating System
I checked the SD-Card (that I purchased from eBay) I noticed the SD-Card had an older NOOBS 2.0.0 installation in an in an invalid subfolder (that’s why it would not boot).
I plugged the Micro SD card into an SD-Card adapter and plugged it into my Mac (time to install a newer OS).
Downloading the latest Raspian Operating System
I visited https://www.raspberrypi.org/downloads/raspbian/ and download raspbian
I downloaded a 1.66GB Zip File.
The file took 30 mins to download.
I expanded the zip file to a 4.66 img file.
Formatting the SD-Card
I formatted my SD Card with the SD Card Associations SD formatting tool from https://www.sdcard.org/downloads/formatter_4/ here.
Setting up the SD-Card
The Raspberry Pi Foundation has a great guide on how to install the Raspberry Pi Raspian image onto an SD-Card here.
I downloaded the free 64MB Etcher program for Mac OS here https://etcher.io/ in order to copy (flash) Raspian onto the SD-Card.
I opened Etcher and selected the Raspian Jesse image file and clicked flash.
The flash will take between 5-10 minutes.
SD Card Contents
The Raspian image is now ready for reinsertion back into the Raspberry Pi to setup
1st Raspberry Pi (Raspian) boot.
The raspberry pi does, in fact, have a power/activity led with the latest operating system 🙂
The desktop loaded quite swiftly too. My Bay special keyboard detected just fine. You can manually connect to your wifi by entering the following command in the terminal (thanks to this guide).
|
1 |
sudo nano /etc/network/interfaces |
Generic Config
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
sudo rasps-config # > I changed the password. # > I chnaged the hostname. # > Configured the pi to book to console (logged in) and not the desktop # > Configured the pi to wait for the network on boot # > Configured the splash screen not to show at boot # > Configured the local to be "en_AU.UTF-8 UTF-8" ("en_AU.UTF-8") # > Configured timezone to be Australia/Sydney # > Configured the keyboard to be a "Generic 104 Key PC" (Englist (US) # > Configured Wifi Country to be AU # > Configured the (Interface) Camera = ON # > Configured the (Interface) SSH = ON # > Configured the (Interface) SPI = ON # > Configured the (Interface) I2C = ON # > Configured the (Interface) Serial = ON # > Configured the (Interface) 1-Wire = ON # > Configured the (Interface) Remote GPIO = OFF # > Configured the (Interface) SPI = ON # > Configured the (Advanced) GPU Memory = 16 # Finish and Reboot |
I am not using the desktop much so I am happy setting 16MB for the video card.
Get the Mac Address and set a static IP Address (to allow remote SSH management)
After a reboot, I was logged into a shell and was able to bring up the devices network details (including mac address).
|
1 |
ifconfig |
Test the network connection
I was successful;y able to ping a remote server via wifi.
|
1 |
ping www.fearby.com |
Static IP Address
Todo: I set up my local ADSL router to give the Raspberry Pi a Static (known) IP every time it boots.
Update the Pi Software (after the network is up)
Now that I knew the network is working I was able to update the Pi software.
|
1 2 3 4 |
sudo apt-get update sudo apt-get upgrade sudo rpi-update sudo reboot |
I re-ran the update just in case. Now my Pi is all up to date.
Installed Misc Software
|
1 2 3 4 |
sudo apt-get install image-magick sudo apt-get install nginx sudo /etc/init.d/nginx start sudo apt-get install php5-fpm |
More on setting up a NGINX web server here.
Set the Pi Mouse (regular sized USB devices)
I had small micro USB to larger USB adapters (from eBay) but could only use a keyboard or mouse at any one time. I Also had a micro USB to multiple micro USB and Ethernet adapter (from eBay) (SN: YS-LAN38) but it failed to work with NOOBS 2.0.0 (maybe it will work with Raspian latest)
I wanted to test regular devices in the case of an emergency.
Help
40 Useful Raspberry Pi Commands.
Todo
Add information on adding and using a 5MP Raspberry Pi camera and setting this up as a http://fearby.com/article/raspberry-pi-2b-security-webcam/secuerity webcam.
Donate and make this blog better
Ask a question or recommend an article
v1.2 DRAFT: Added more images
Creating and configuring a CentOS server on Digital Ocean
Normally I prefer Ubuntu servers but I needed a CentOS Server for some software that requires it.