Code, Security and Server Stuff

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

"If you're not still learning, you're already dying."
- Ryan Holiday - Ego is the Enemy

Follow me on Twitter: @FearbySoftware

View All Posts.

terminal

Useful OSX Terminal Commands

by

Following on my from Useful Linux Terminal Commands here is my Useful OSX Terminal Commands. This guide is handy for setting up a development OSX installation (e.g Apache, PHP, MySQL etc). I will update this post when I find new commands.

Advertisement:


First, read my Useful Linux Terminal Commands post.

Normally I set up Ubuntu servers in the cloud on Vultr (read my guide here) or Ubuntu Servers on Digital Ocean (read my guide here) but this is how I can quickly set up a development environment locally on OSX.

Useful OSX Commands

Prevent your Mac from falling asleep

Prevent your Mac from falling asleep (for 1 minute)

Check Path

$PATH
/Library/Frameworks/Python.framework/Versions/3.4/bin:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands

Compare files

Show hidden files in Finder

Find file on OSX

Find all listening ports

Find process listening on port 80

Find a Process

Find A File (e.g httpd.conf)

Note: OSX Apache Index file location is…

To edit the default HTML file

Built-In Apache

Disable built-in OSX apache daemon

Restart built-in Apache daemon

View other startup daemons

View All Startup Daemons

Force network drives to automatically connect after each login (instead of manually connecting)

More: https://www.tekrevue.com/tip/automatically-connect-network-drive/

Set PHP as default Web Server delivered file in local Apache

Set…

Install PHP 7.0 on to OSX

FYI: 7.1 and 7.2 are out but I find 7.0 is more stable with WordPress (read more here).

More: https://php-osx.liip.ch/

Create a diag file in your OSX Default Apache root

Edit your default index file

Contents

Now load the file

Debug Info

Install MySQL on OSX

Download: https://dev.mysql.com/downloads/mysql/

Download MySQL

Help: https://dev.mysql.com/doc/refman/5.7/en/osx-installation-pkg.html

Uninstall MySQL OSX
Read: https://community.jaspersoft.com/wiki/uninstall-mysql-mac-os-x

Add MySQL to PATH

Reboot and Check the MySQL Service

Service Status

Check MySQL Version

Login to MySQL from the Command Line

Set a new root password (after logging in as root)

fyi: You can obtain a good password here.

Bonus: Install MySQL (PHP) management tool  (in Apache)

Download: https://www.adminer.org/#download Save the file to /Library/WebServer/Documents/adminer.php

Load Adminer in your browser and log in with your root password

Login to Adminer (via Apache/PHP)

Adminer

Creating MySQL Tables in Adminer (via Apache/PHP)

Create Table

Adminer allows you to fully manage your MySQL (create databases, users, run queries, alter objects etc)

Adding Records to MySQL with Adminer

Adding Records

Remove MySQL:

Read more here: https://community.jaspersoft.com/wiki/uninstall-mysql-mac-os-x

Get your Mac Processor Name

Show All OSX System Settings

Command:

Show Live File System Usage

Show last 10 lines of the system log

Get Your Local WAN IP Address

Get your Public IPV4 Address

More to come.

Hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

v1.2 Find file (current version)

v1.1 Removed system info chunk (too big to format)

Useful Linux Terminal Commands

by

Below are Ubuntu Linux commands I use often to setup, debug maintain servers.

Advertisement:



Read this guide for Useful OSX Commands (for setting up Apache, PHP, MySQL, Adminer etc on OSX)

I recently moved my domain from a C-Panel hosted domain (and Email to Google G Suite (my guide here)) to a self-managed Digital Ocean domain (my LetsEncrypt Guide here, my Digital Ocean guide here, my AWS setup guide here, my Vultr setup guide here) and needed to transfer my WordPress site. Setup your own Digital Ocean Ubuntu server form $5  a month (get the first 2 months free by clicking here) or setup your own Vultr Ubuntu server for as low as $2.5/month by clicking here.

How to Reboot (from time to time when prompted with *** system restart required ***” messages appear).

How to set up a  bash file (*.sh) as Executable.

The file will now be executable.

Viewing your crontab (Windows Task Scheduler equiv)

Ping a port

Rename a folder

Set the owner of a folder

Check the rsync port

Run a file every 1 minute

Show server name

How to verify patch status for Meltdown and Spectre

Read my guide here to install the patch here.

Verify Spectre and Meltdown patch status

or

Restart network

More here.

Show Operating System Name

hostnamectl ! grep "Operating System"
 Operating System: Ubuntu 10.04.3 LTS

Show installed Packages

Show all packages with “PHP” in the name.

Send Messages Other Logged In Users (CLI)

Show all user logged in

Sent a message to usr1

Sent a message to the root console

Messages will appear at the bottom of the user’s console.

Processor

List processes in a tree view (… = removed)

View major processes by usage/memory

Read more on ps here.

How big is a folder

Change FIle Create/Modify time’s

Change File Creation Date

Change file modify/accessed time

Tree

Tree needs to be installed first

Show an ASCII representation of a folder structure

Show files in a  structure

Show directories

List all files and folders in a structure

Backup a  www folder

Common (Digital Ocean) Debugging commands

Networking

Display all TCP connections

Display all UDP connections

List all Listening Connections

Show all Network stats

Show all UDP Network stats

Show all TCP Network stats

Show network packets

Displaying raw info

Show open ports

Upload a file to a remote server over ssh

Zip files

Install zip

Zip a  whole directory (with high compression)

Zip a  whole directory (excluding *.tmp, *temp, *.bak and *.zip file types)

Zip Help

Backup NGINX

Unzip

Installing Unzip

Unzip a  file

unzip filename.zip

Updates

Setup Unattended Ubuntu Security updates

At login, you should receive

Show Configured NGINX server names

Services

Reload daemon services

Verify the MongoDB service file exists

View the status of services

Local Dump

Show All Defined Ports

Show defined rsync ports

Show listening ports (active connections)

Show service status

View your bash history for a past command (e.g “openssl”).

grep "openssl" ~/.bash_history
openssl req -new -newkey rsa:4096 -nodes -keyout fearby.key -out fearby.csr

View last 1 line of a file.

Ping a server

How to check system uptime (and load average)

Uptime in friendly format

The load averages at the end are the last 1, 5 min and 15 minutes.

The w command is handy for showing uptime information as-well as logged in users (The ‘w -i’  parameter -i is handy for seeing what IP people are logged in from).

How to download a file

How to edit NGINX configuration

sudo nano /etc/nginx/nginx.conf

and

How to find a file

also

locate php.ini

Find contents in files (recursive)

Find files by name and run a command on each

Show differences in files

Show contents of file (e.g urls.txt)

cat urls.txt
http://www.server1.com
http://www.server2.com
http://www.server3.com
http://www.server4.com
http://www.server5.com

Download all files mentioned in a text file.

View all packages with updates

Output:

or (on ubuntu 16.04)

Updates

Always backup your server’s configuration before updating.

Backup MySQL

Crontab

Add this to crontab -e to backup at 1 am every day

/scripts/shrinkmysql.sh  script to shrink SQL files

I had to add this to crontab

Cron job to shrink SQL dumps at 2am every day

Write to a single log file from the crontab at 2 am every day.

todo

Query a package (e.g. siege package)

How to set up a free SSL certificate (see my guide here).

How to set your timezone.

How to restart PHP

How to show the time on the server

sudo hwclock --show

Reload and restart the NGINX configuration and web server.

JSON viewing program

Installing

Using

or

output from json tool

Below are utilities I use a lot.

ncdu file size utility

Installing

Using

pydf disk checking utility

Installing

Using

output from pydf tool

ntp timezone service

Installing

Using

Displaying startup processes

Installing

Using

htop process manager.

Installing

Using

output from htop tool

Network Benchmarking (between two servers)

I use iperf to measure total bandwidth between two servers. You will need to allow port 5001 (TCP IPV4 and TCP IPV6 in and out) in any local firewalls and hosts GUIs.

Allow port 5001 on an ufw firewall (IN and OUT)

Deny IP

Allow port 22 access to known IP

Deny Outgoing Port

Allow out on port to known IP

More securing Ubuntu in the cloud here.

Install iperf on the target and source Ubuntu server.

Run this on the listening server.

Run this on benchmarking server (and add the IP of the listening server).

iperf -c 123.123.123.123

Results

Screen dump of ipref -c ip
iperf benchmarking output

Testing concurrent connections to a web server with siege.

Install siege

Benchmarking a HTTP server

./siege -t1m c10 'https://fearby.com'

#10 Concurent users

Benchmarking HTTPS sites

Install siege 4.0.2 ( steps here ).

Verify siege 4.0.2 is installed by running

Now can benchmark https sites

View incoming connections on the target server

I always increase my history size and tell it not to store duplicates.

Viewing your typed terminal history

Increasing your history size

How to Update Ubuntu

or

Edit SSH authorized keys

Search file and show lines where text matches

View packages with updates

View Boot Text

Automatic Monitoring (ever 1 second)

Active network connections

Network Packets

Free memory

Memory breakdown

or

Monitor NGINX memory

more soon…

Read this guide for Useful OSX Commands (for setting up Apache, PHP, MySQL, Adminer etc on OSX)

50 most useful Linux commands (view here).

View other Linux command informing sites here, here and here.

 

Donate and make this blog better




Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

v1.8 Show users and senda messages to other users

v.71 Show Operating System Name

v1.7 added Meltdown and Spectre patch information

v1.6 Removed -x fom zip directory

V1.5 Restart Network

v1.4 added Useful OSX Commands

Connecting to an AWS EC2 Ubuntu instance with Cloud 9 IDE as user ubuntu and root

by

Recently I setup and Amazon EC2 Ubuntu Server instance and wanted to connect it to the awesome Cloud 9 IDE. I was sick of interacting with a server through terminal windows.

Advertisement:



Use this link and get $19 free credit with Cloud 9: https://c9.io/c/DLtakOtNcba

c9io15-004

Cloud 9 IDE (sample screenshot)

C9 IDE

Previously I was using Digital Ocean (my Digital Ocean setup guide here) and this was simple, you get a VM and you have a root account and you do what you want.  Amazon AWS however, have extra layers of security that prevent logging in as root via SSH and that can be a pain with Cloud 9 as your workspace tree is restricted to the ~/ (home) folder.

Below are the steps you need to connect to an AWS instance with user “ubuntu” and “root” with Cloud 9.

Connecting to an AWS instance with Cloud 9 as user “ubuntu”

1. Purchase and set-up your AWS instance (my guide here).

2. You need to be able to login to your AWS server from a terminal prompt (from OSX).  This may include opening port 22 the AWS Security Group panel. Info on SSH logins here.

3. On your AWS server (from step 2) Install NodeJS.

You will know node is installed if you get a version returned when typing the following bash command.

node -v

tip: If node is not installed you can run the Cloud 9 pre-requisites script (that includes node).

curl -L https://raw.githubusercontent.com/c9/install/master/install.sh | bash

4. Ensure you have created SSH key on Cloud 9 (guide here).

5. Copy your Cloud 9 SSH key to the clipboard.

6. On your AWS server (in step 2) edit the ~/.ssh/authorized_keys file and paste in the Cloud 9 SSH key (after you AWS key pair that was added from the setup of AWS) to a new line and save the file.

7. Log in to Cloud 9 and click create Workspace then Remote SSH Workspace.

  • Name your workspace (all lowercase and no spaces).
  • Username: ubuntu
  • Hostname: Add your AWS ec2 server hostname.
  • Initial Path: This has to be ~/

c9io15-000

8. Click Create Workspace

c9io15-002

9. If all goes well you will have a prompt to install the prerequisites.

c9io15-001

If this fails check out the Cloud 9 guide here.

Troubleshooting: I had errors like “Project directory does not exist or is not writable and “Unable to change File System Path in SSH Workspace” because I was trying to set the workspace path as “/” (this is not possible on AWS with the “ubuntu” account.

10. Now you should have a web-based IDE that allows you to browse your server, create and edit files, run termials instances that will reconnect if your net connection or browser tab drops out (you can even go to a different machine and continue with your session).

c9io15-003

Connecting to an AWS instance with Cloud 9 as user “root

Connecting to your server as the “ubuntu” server is fine if you just need to work in your “ubuntu” home folder.  As soon as you want to start changing other settings outside of your home folder you are stuck.  Granting “ubuntu” higher privileges server wide is a bad idea so here is how you can enable “root” login via SSH access.

WARNING: Logging in as ROOT IS BAD, you should only allow Root Login for short periods and it is advisable to remove root login abilities as soon as you do not need them or in production.

Having root access while developing or building a new server saves me bucket loads of time so lets allow it.

1. Follow step 1 to 5 in the steps above (setup AWS, ssh access via terminal, install node, create cloud 9 ssh key, copy the cloud 9 ssh key to the clipboard).

2. SSH to your AWS server and edit the following file:

Save.

3. Backup your root authorised keys file

4. Edit the root authorized_keys file and paste in your Cloud 9 SSH Key.

c9io15-005

Advertisement:



5. Now you can create a Cloud 9 Connection to your server with root

  • Name your workspace (all lowercase and no spaces).
  • Username: root
  • Hostname: Add your AWS ec2 server hostname.
  • Initial Path: This has to be /

c9io15-007

tip:  If you have not added you SSH key correctly you will receive this error when connecting.

c9io15-006

6. You should now be able to connect to AWS ec2 instances with Cloud 9 as root and configure/do anything you want without switching to shell windows.

c9io15-009

Security

As a precaution, do check your website often in https://www.shodan.io and see if it has open software or is known to hackers. 
Enjoy

If this guide has helped please consider donating a few dollars.

Donate and make this blog better





Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

 V1.6 security