Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
push
Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
This is a draft post showing how you can monitor the performance of a server (or servers) with NixStats and receive alerts by SMS, Push, Email, Telegram etc
fyi: This is not a paid post, this is just me using the NixStats software to monitor my servers and send alerts.
Advertisement:
Finding a good host
If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.
Buy a domain name here
Monitoring Servers
The post below will show you how you can monitor servers online with https://nixstats.com/ and send alerts when resources reach limits or servers fail.
I signed up and started a Nixstats (14 day free trial).
After I created an account I was emailed by Nixtsats with agent install instructions for Linux (1 line). I was also advised to add contacts and to set up alerts.
I logged into the Nixstats settings and set up…
- My Timezone
- Default reporting period
- First name and Surname
- Reporting emails
- etc
Nixstats Subscription Upgrade
Subscription options
- Free (5 monitors, 1 server, 24-hour data retention etc)
- Founder (25 monitors, 10 servers, 30-day data retention etc)
- Business (100 monitors, 15 servers, 60-day data retention etc)
I enabled the limited founder subscription so I can monitor 10x servers (this deal is too good to miss). I tried creating a status page myself last year and it is terribly hard.
I am now out of the free trial period 🙂 Let’s start monitoring many servers.
I enabled two factor Auth to Nixstats logins
I created a Nixstats API key for future use (watch this space)
I installed the Nixstats agent (the dashboard gave a 1 line command you can run as root to install the agent (on Linux)).
FYI: Command (######################## is a number linked to your account)
1 | wget --no-check-certificate -N https://www.nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh ######################## |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | wget --no-check-certificate -N https://www.nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh ######################## --2018-10-02 09:53:56-- https://www.nixstats.com/nixstatsagent.sh Resolving www.nixstats.com (www.nixstats.com)... 2400:cb00:2048:1::6819:8013, 2400:cb00:2048:1::6819:8113, 104.25.128.19, ... Connecting to www.nixstats.com (www.nixstats.com)|2400:cb00:2048:1::6819:8013|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 38708 (38K) [application/octet-stream] Saving to: 'nixstatsagent.sh' nixstatsagent.sh 100%[====================================================>] 37.80K --.-KB/s in 0.1s 2018-10-02 09:53:56 (338 KB/s) - 'nixstatsagent.sh' saved [38708/38708] Found Ubuntu ... Installing ... Installing Python2-PIP ... Installing nixstatsagent ... Generation a server id ... Got server_id: ###################### Creating and starting service Created symlink /etc/systemd/system/multi-user.target.wants/nixstatsagent.service -> /etc/systemd/system/nixstatsagent.service. Created the nixstatsagent service |
Server Dashboard
below is a summary of all connected servers ( https://nixstats.com/dashboard/servers ).
Monitor Setup
I set up a number of monitors to monitor ping replies and https traffic
Advanced Monitoring
I can also set the monitor credentials, timeouts, retries, auth methods, max redirects and frequency. If you server blocks login or resource GET attempts you may need to whitelist IP’s. IP’s of monitoring servers are located here https://nixstats.com/whitelist.php
Monitor Summary
The default dashboard is very informative. Feel free to create your own dashboards that focus on your own infastructure or apps.
Individual Server Reports
Advertisement:
You can click on a server and monitor it in detail.
Server Memory Graphs
Long-term memory graphs.
Install Optional Nixstats Plugins
Nixstats offers many plugins to monitor software that is installed on your server (e.g NGINX, MySQL, PHP etc).
1) NGINX Monitoring (Plugin)
To enable NGINX monitoring I read https://help.nixstats.com/en/article/monitoring-nginx-50nu7f/
I edited my NGINX sites-enabled config.
1 | sudo nano /etc/nginx/sites-enabled/default |
I added the following
1 2 3 4 5 6 7 8 9 | server { listen 127.0.0.1:8080; server_name localhost; location /status_page { stub_status on; allow 127.0.0.1; deny all; } } |
I tested, reloaded and restarted NGINX
1 2 3 | nginx -t nginx -s reload /etc/init.d/nginx restart |
The status page will only be available on the local machine, I tested the page on the local machine
1 2 3 4 5 | wget -qO- http://127.0.0.1:8080/status_nginx Active connections: 3 server accepts handled requests 15 15 31 Reading: 0 Writing: 1 Waiting: 2 |
It’s Working.
I edit /etc/nixstats.ini
1 | sudo nano /etc/nixstats.ini |
I remove comments before these lines to enable the plugin
1 2 3 | [nginx] enabled = yes status_page_url = http://127.0.0.1:8080/status_nginx |
I ran the following command to see if NGINX monitoring is possible
1 | nixstatsagent --test nginx |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 | nginx: { "accepts": 39, "accepts_per_second": 0.0, "active_connections": 6, "handled": 39, "handled_per_second": 0.0, "reading": 0, "requests": 119, "requests_per_second": 0.0, "waiting": 5, "writing": 1 } |
It’s Working
I restarted the nixstatsagent
1 | service nixstatsagent restart |
I can now view NGINX properties like active_connections in my dashboard. 🙂
2) Enable PHP-FPM Monitoring (Plugin)
Looks like a PHP-FPM monitoring was recently added lets set that up too. Read my guide on setting up PHP child workers here.
We’ve added a premade dashboard for PHP-FPM. If you’re not yet monitoring PHP-FPM take a look at the integration guide https://t.co/X4ywRHw9hX pic.twitter.com/aag1fTsr3R
— Nixstats (@nixstats) September 6, 2018
To enable PHP-FPM monitoring I read https://help.nixstats.com/en/article/monitoring-php-fpm-1tlyur6/
I edited my PHP-FPM ini file
1 | sudo nano /etc/php/7.2/fpm/pool.d/www.conf |
I added the following line
1 | pm.status_path = /status_phpfpm |
Restart PHP
1 | sudo service php7.2-fpm restart |
I added the following to /etc/nginx/sites-enabled/default localhost server block added above Note I use php 7.2 below (read more here).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | server { listen 127.0.0.1:8080; server_name localhost; location /status_phpfpm { access_log off; allow 127.0.0.1; deny all; include fastcgi_params; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_pass unix:/run/php/php7.2-fpm.sock; log_not_found off; } } |
I tested, reloaded and restarted NGINX
1 2 3 | nginx -t nginx -s reload /etc/init.d/nginx restart |
I restart PHP-FPM
1 | sudo systemctl restart php7.2-fpm |
Enabled the plugin in /etc/nixstats.ini
1 2 3 | [phpfpm] enabled = yes status_page_url = http://127.0.0.1:8080/status_phpfpm?json |
I tested the status page
1 | wget -qO- http://127.0.0.1:8080/status_phpfpm?json |
Output:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | { "pool":"www", "process manager":"static", "start time":1538654543, "start since":178, "accepted conn":28, "listen queue":0, "max listen queue":0, "listen queue len":0, "idle processes":49, "active processes":1, "total processes":50, "max active processes":2, "max children reached":0, "slow requests":0 } |
Advertisement:
I tested the agent
1 | nixstatsagent --test phpfpm |
Output:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | phpfpm: { "accepted_conn": 51, "accepted_conn_per_second": 0.0, "active_processes": 1, "idle_processes": 49, "listen_queue": 0, "listen_queue_len": 0, "max_active_processes": 2, "max_children_reached": 0, "max_listen_queue": 0, "pool": "www", "process_manager": "static", "slow_requests": 0, "start_since": 318, "start_time": 1538654543, "total_processes": 50 } |
I can now query PHP-FPM status values in Nixstats 🙂
Enable MySQL Monitoring (Plugin)
To enable MySQL monitoring I read https://help.nixstats.com/en/article/monitoring-mysql-1frskd8/
I edited the nixstats.ini
1 | sudo nano /etc/nixstats.ini |
I enabled the mysql section in nixstats.ini and added my mysql credentials
1 2 3 4 5 6 7 8 | [mysql] enabled=yes username=mysqluser password=####################### host=127.0.0.1 database=mysql port=3306 socket=null |
I ran this command to test MySQL querying
1 2 3 | nixstatsagent --test mysql mysql: Load error: No module named MySQLdb |
I had an error.
A quick Google revealed I had to install a mysql python module.
1 | sudo apt-get install python-mysqldb |
Allow localhost to connect to MySQL
Edit /etc/mysql.cnf and allow all localhost and external connections (I could not bind to localhost and an external IP at the same time)
1 | bind-address = 0.0.0.0 |
TIP: Ensure you have firewalled access to your MySQL server, never open it up without protection.
Let’s try again
1 | nixstatsagent --test mysql |
Output
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | mysql: { "aborted_clients": 0, "aborted_connects": 0, "binlog_cache_disk_use": 0, "binlog_cache_use": 0, "bytes_received": 0, "bytes_sent": 0, "com_delete": 0, "com_delete_multi": 0, "com_insert": 0, "com_insert_select": 0, "com_load": 0, "com_replace": 0, "com_replace_select": 0, "com_select": 0, "com_update": 0, "com_update_multi": 0, "connections": 0, "created_tmp_disk_tables": 0, "created_tmp_files": 0, "created_tmp_tables": 0, "key_read_requests": 0, "key_reads": 0, "key_write_requests": 0, "key_writes": 0, "max_used_connections": 3.0, "open_files": 14.0, "open_tables": 316.0, "opened_tables": 0, "qcache_free_blocks": 1.0, "qcache_free_memory": 16760152.0, "qcache_hits": 0, "qcache_inserts": 0, "qcache_lowmem_prunes": 0, "qcache_not_cached": 0, "qcache_queries_in_cache": 0, "qcache_total_blocks": 1.0, "questions": 0, "select_full_join": 0, "select_full_range_join": 0, "select_range": 0, "select_range_check": 0, "select_scan": 0, "slave_open_temp_tables": 0.0, "slow_launch_threads": 0, "slow_queries": 0, "sort_range": 0, "sort_rows": 0, "sort_scan": 0, "table_locks_immediate": 0, "table_locks_waited": 0, "threads_cached": 2.0, "threads_connected": 1.0, "threads_created": 0, "threads_running": 1.0, "uptime": 35837.0 } |
Nice,
I restart MySQL
1 | sudo systemctl restart mysql |
I restart my Nixstats service
1 | service nixstatsagent restart |
Now let’s monitor MySQL in Nixstats
I can now view MySQL metrix
Status Page
Nixstats allows you to create a status page ( https://nixstats.com/pages/overview ) where you can add any servers or monitors to that page. This stats page is truly awesome, it builds a live status page based on data coming from your installed agents.
You can even set up a custom subdomain that points to a Nixstats hosted status page too (e.g https://status.yourdomain.com).
FYI: An SSL certificate on your staus page may take a few hours to set up. Don’t panic if it is not instantatly available.
Nice.
This saves doing it yourself. The status page will look like it running on your server.
Status Page
You can create a status page that automatically aggregates collected data from monitors and displays them in a nice layout.
This is great, I used to do my own status pages but not anymore.
Alerts
I added a contact so I could receive alerts. I could then add my mobile, email and PushOver key (to receive push notifications) and Telegram Bot API token.
Test Alerts
I sent a test alert to each service against the contact.
Advertisement:
I activated a Pushover licence on my Android device for about $7.49 AUD (one off) to ensure I keep getting Push Notifications.
Nixstats have links that show you how you can create a Telegram Bot and Pushover.net account.
Pushover will cost about $5 USD one off per device (see faq).
I created the following alerts
Alert: Disk Usage higher than 90%
Alert: Load greater than 90 per cent for 1 minute
Alert: Less than 5 percent memory free.
Summary of alerts.
I also added a CPU reached 95% one for 5 mins alert too (but it’s not pictured above)
I forgot to specify alert recipients and methods for each alert so I edited each alert and added the contact.
Now it’s time to test the alerts.
I shut down a server to test alerts
1 | shutdown -h now |
Alerts to my defined Email, SMS, Push and Telegram are working a treat 🙂
After I rebooted the server I also received alerts about the server being back up.
The status page showed the server that was offline too.
Nice
Troubleshooting
I had an issue instaling the agent on Debian
I ran the following command
1 2 3 4 5 6 7 8 9 10 | wget --no-check-certificate -N https://www.nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh ####################### --2018-10-02 00:41:38-- https://www.nixstats.com/nixstatsagent.sh Resolving www.nixstats.com (www.nixstats.com)... 2400:cb00:2048:1::6819:8113, 2400:cb00:2048:1::6819:8013, 104.25.129.19, ... Connecting to www.nixstats.com (www.nixstats.com)|2400:cb00:2048:1::6819:8113|:443... connected. HTTP request sent, awaiting response... 304 Not Modified File 'nixstatsagent.sh' not modified on server. Omitting download. nixstatsagent.sh: line 508: [: Installer exited with error code 0. See nixstatsagent.log for details.: integer expression expected An error occured, please check the install log file (nixstatsagent.log)! |
Contents of nixstatsagent.log
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | cat nixstatsagent.log Ign:1 http://deb.debian.org/debian stretch InRelease Hit:2 http://deb.debian.org/debian-security stretch/updates InRelease Hit:3 http://deb.debian.org/debian stretch-updates InRelease Hit:4 http://deb.debian.org/debian stretch Release Ign:5 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic InRelease Ign:7 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic Release Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages Hit:9 https://packages.sury.org/php stretch InRelease Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages Err:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages 404 Not Found Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en Reading package lists... W: The repository 'http://ppa.launchpad.net/ondrej/php/ubuntu cosmic Release' does not have a Release file. E: Failed to fetch http://ppa.launchpad.net/ondrej/php/ubuntu/dists/cosmic/main/binary-amd64/Packages 404 Not Found E: Some index files failed to download. They have been ignored, or old ones used instead. nixstatsagent.sh: line 118: apt-get upgrade returned error code 100. Please see nixstatsagent.log for details.: command not found |
I asked the Nixstats chat help and I was advised I had a dead repository (I removed this (editing the dead repo in the appropriate file in /etc/apt/) and all was ok)
I had trouble testing my Telegram alerts but it was my fault as I forgot to follow the bot account I created. Telegram does not allow message from a user (bot) unless you follow them.
A chat with the Nixstats staff sorted me out. Thanks, Nixstats chat team.
I had an issue with a missing python mysql package
1 | Load error: No module named MySQLdb |
I solved it by instaling python-mysqldb
1 | sudo apt-get install python-mysqldb |
Nixstats Help
Nixstats have a help subdomain: https://help.nixstats.com/en/
Error Logs Plugin
I did ask Nixstats on Twitter and they said they are working on a logging plugin, I can’t wait for that.
We’re launching a closed beta for Logging at Nixstats. Contact us to get setup! You can search and tail log files across all your servers! pic.twitter.com/FIeip2SOUw
— Nixstats (@nixstats) October 4, 2018
I now have access to beta log features and can see log tabs in Nixstats
I had or check the version of my rsyslogd
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | rsyslogd -v rsyslogd 8.32.0, compiled with: PLATFORM: x86_64-pc-linux-gnu PLATFORM (lsb_release -d): FEATURE_REGEXP: Yes GSSAPI Kerberos 5 support: Yes FEATURE_DEBUG (debug build, slow code): No 32bit Atomic operations supported: Yes 64bit Atomic operations supported: Yes memory allocator: system default Runtime Instrumentation (slow code): No uuid support: Yes systemd support: Yes Number of Bits in RainerScript integers: 64 |
I edited: /etc/rsyslog.d/31-nixstats.conf
I pasted
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | ########################################################## ### Rsyslog Template for Nixstats ### ########################################################## $WorkDirectory /var/spool/rsyslog # where to place spool files $ActionQueueFileName fwdRule1 # unique name prefix for spool files $ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) $ActionQueueSaveOnShutdown on # save messages to disk on shutdown $ActionQueueType LinkedList # run asynchronously $ActionResumeRetryCount -1 # infinite retries if host is down template(name="NixFormat" type="string" string="<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [5bb8825c3f0bc16aa068f144@5fJR7vuYTJOX6cEiyZlLv5By tag=\"rsyslog\"] %msg%\n" ) action(type="omfwd" protocol="tcp" target="log.nixstats.com" port="514" template="NixFormat") #################END CONFIG FILE######################### |
I restarted the rsyslog service
1 | sudo service rsyslog restart |
Live Log Output
I can see a live log from (unknown) logs.
I can see the firewall blocking access to certain ports.
Search logs
Conclusion
This is one of the best software packages I have seen in a while. I have developed status pages in the past (no more). Great work Nixstats.
Please check out Nixstats today, they are awesome. Signup for a free account and consider the limited time founder plan (it’s a bargain).
Nixstats Live chat support is awesome
Server Plug
If you need a server, consider using my referral code and get $25 UpCloud VM credit if you need to create a server online.
https://www.upcloud.com/register/?promo=D84793
Advertisement:
Ask a question or recommend an article
Revision History
v1.2 Logs beta
v1.1 Logging Tweet
v1.0 Initial Post
Setting up GitHub on an iMac and creating a project
Here is my quick guide on setting up new GitHub repository on an iMac adding files and syncing with a repository. Read the GitHub 101 if you are a beginner. Read my BitBucket guide here.
Advertisement:
Why
Using git allows you to make backups, use code versioning, rollback code, use multiple development machines/have multiple users, auto merging code etc.
Github can be cheaper, more flexible and offer more features than something like Dropbox.
How
Create an account at GitHub and Login ( https://github.com/login ).
Don’t forget to set up Two Factor Authentication for extra security.
Login to GitHub (signup for a free account if need be).
Enter your Two Factor Authentication code every time you login.
Before you can read or write to your Git repository you need to set up a link between your computer (Mac) and GitHub by Setting up a Secure SSH Key with GitHub. About SSH.
If you have read any of my guides on setting up servers (e.g The quickest way to setup a scalable development IDE and web server, How to buy a new domain and SSL cert from NameCheap, a Server from Digital Ocean and configure it or Connecting to an AWS EC2 Ubuntu instance with Cloud 9 IDE as user ubuntu and root ) you will be familiar with SSH keys.
You can check your existing SSH Keys by typing the following.
1 | ls -al ~/.ssh |
Follow this GitHub guide on creating a new SSH key to use with GitHub.
I found I had to do the following in the Mac terminal to get this to work.
1 2 3 4 5 6 7 8 9 | eval "$(ssh-agent -s)" mkdir /gitrsatemp cd /gitrsatemp sudo ssh-keygen -t rsa -b 4096 -C "simon@fearby.com" # specify output as /gitrsatemp/githubrsa # used a passphrase from https://www.grc.com/passwords.htm mv /gitrsatemp/* ~/.ssh/ cd ~/.ssh/ ssh-add -K ./githubrsa |
You can now add your SSH key to GitHub.
1 2 | cd ~/.ssh/ sudo pbcopy < ./githubrsa.pub |
You can then add your public key to https://github.com/settings/keys
Add the SSH Key.
You can then test the SSH connection to GitHub.
At first, I tried this but got an error.
1 2 3 4 5 | sudo ssh -T git@github.com The authenticity of host 'github.com (192.30.255.113)' can't be established. RSA key fingerprint is SHA256:key_redacted. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'github.com,192.30.255.113' (RSA) to the list of known hosts. |
This is what I got when I added my SSH Key to my GitHub account. This worked for me (this updated my known_hosts file).
1 2 | sudo ssh -T git@github.com Hi SimonFearby! You've successfully authenticated, but GitHub does not provide shell access. |
Tip: Don’t forget to save adding your SSH Key to GitHub
If you need help contact GitHub.
GitHub Pricing
GitHub offers free public repository pricing or $7 a month for unlimited personal repositories for personal use. Bitbucket is the Atlassian owned alternative to GitHub, they have free repositories for up to 5 users with 1TB.
Read my Setting up BitBucket on an iMac and creating a project guide.
Read on the differences between GitHub and BitBucket.
Create your first test public Repository.
Load https://www.github.com and click Start a Project, or go to https://github.com/new
To specify a Readme file and an ignore file if need be (we can do this later). You will need to be a paid Git member to create private repositories.
When your repository is created it will be ready to upload files: https://github.com/SimonFearby/testgitproject001
Creating a local folder
Create a local folder for the repository etc.
1 2 3 4 5 6 7 8 9 10 | # create a folder on your desktop cd ~/Desktop/ mkdir testgitproject001 cd testgitproject001/ # Check the git status git status fatal: Not a git repository (or any of the parent directories): .git # the folder is now ready files from the repository. |
Liking the remote repository to the local folder.
First, we need to initialize git locally by typing the following.
1 2 | git init > Initialized empty Git repository in /Users/simon/Desktop/testgitproject001/.git/ |
Now we can add the remote “origin” repository to the local repo.
1 | git remote add origin git@github.com:SimonFearby/testgitproject001.git |
We can now see connected remote repositories by typing the following
1 2 3 | git remote -v origin git@github.com:SimonFearby/testgitproject001.git (fetch) origin git@github.com:SimonFearby/testgitproject001.git (push) |
Syncing (Pulling) the Repository
Generally, you want to pull all files from the remote repository after you create it (and the readme file).
1 | git pull origin master |
This worked 🙂
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | # What directory are we in pwd /Users/simon/Desktop/testgitproject001 # No files ls -al total 0 drwxr-xr-x 3 simon staff 102 28 Jun 00:19 . drwx------+ 53 simon staff 1802 28 Jun 00:00 .. drwxr-xr-x 10 simon staff 340 28 Jun 00:21 .git # Let's get the repo files git pull origin master remote: Counting objects: 4, done. remote: Compressing objects: 100% (3/3), done. Unpacking objects: 100% (4/4), done. remote: Total 4 (delta 0), reused 0 (delta 0), pack-reused 0 From github.com:SimonFearby/testgitproject001 * branch master -> FETCH_HEAD * [new branch] master -> origin/master # Is the readme is there. ls -al total 32 drwxr-xr-x 5 simon staff 170 28 Jun 00:30 . drwx------+ 53 simon staff 1802 28 Jun 00:00 .. drwxr-xr-x 13 simon staff 442 28 Jun 00:30 .git -rw-r--r-- 1 simon staff 11357 28 Jun 00:30 LICENSE -rw-r--r-- 1 simon staff 49 28 Jun 00:30 README.md |
Pulling, Pushing, Branches and Forking.
You can read more about pulling, pushing, branches and forking here.
This course helped me https://www.udemy.com/learn-android/
Adding your first local file(s) to the repository
Adding all files to the new directory.
1 | git add -A |
Adding a single file:
1 | git add newfilename.txt |
Here is an example of adding a new file.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | # Create a new file touch newfilename.txt #edit the file. sudo nano newfilename.txt # Tell git we want this file to be managed git add newfilename.txt # What is the status of this repository git status On branch master Changes to be committed: (use "git reset HEAD <file>..." to unstage) |
Set your default commit username (help)
1 | git config --global user.name "Mona Lisa" |
Set your default commit email address (help).
1 | git config --global user.email "email@example.com" |
Double check config here
1 | git config --global --edit |
You will need to use vim to insert then save changes. I like the nano text editor better so I am changing the default editor.
Setting your default as nano (and not vim)
nano (and not vim)
1 | git config --global core.editor nano |
Submitting (pushing) changes
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | # Create a new file locally touch newfilename.txt # edit the local file sudo nanonewfilename.txt # Tell git we want to include it. git addnewfilename.txt # commit the file to the remote (origin) server with a commit message git commitnewfilename.txt -m "addednewfilename.txt" # Merge all changes with the remote (origin) server git push origin master |
The changes have been pushed (merged)
Adding another file.
This is how I added another single file locally and pushed to the remote (origin) server
1 2 3 4 5 6 7 8 9 10 11 12 13 14 | # Create a new file locally touch anothernewfile.txt # edit the local file sudo nano anothernewfile.txt # Tell git we want to include it. git add anothernewfile.txt # commit the file to the remote (origin) server with a commit message git commit anothernewfile.txt -m "added anothernewfile.txt" # Merge all changes with the remote (origin) server git push origin master |
Now we have two new public files in a repository and locally.
README.md and markdown
Now we need to make the README.MD file look nice and reflect the changes.
More info in markdown formatting here, here, here and here. Here is a good markdown table generator.
I made some changes (see) and pushed to the repository (origin).
1 2 3 4 5 6 7 8 | # edit the file sudo nano README.md # Tell git we want to commit the changes git commit README.md -m "edited README.md" # Merge local changes with the remote repository (origin) git push origin master |
My local README.md markdown changes
1 2 3 4 5 6 7 8 9 10 11 12 13 | # testgitproject001 Simon's Test Git Project 001 # *Added* **two** test __text__ files (~~they are not important~~) - [x] newfilename.txt - [x] anothernewfilename.txt - [ ] notmergedlocalfile.txt ## This is for my (GitHub guide on my blog](https://www.fearby.com/article/setting-up-github-on-an-imac-and-creating-a-project/) Inline-style:  |
This is what is looks like on GitHub after I commit and push
You can use pages on GitHub with markdown to create versioned documentation and help.
Commit and check the git status frequently
It is strongly advisable to frequently commit files (to a staging/not master repository) and check local it status for any issues. Commit as frequently as you remember.
1 2 3 4 5 6 7 8 9 10 11 12 13 | # A forgotten local edit sudo nano README.md # Show a gitstatus report git status On branch master Changes not staged for commit: (use "git add <file>..." to update what will be committed) (use "git checkout -- <file>..." to discard changes in working directory) modified: README.md no changes added to commit (use "git add" and/or "git commit -a") |
You can push or delete changes if need be.
Adding many subfolders and many files.
Let’s say you have just added three subfolders (iOS Project Code (12 files), Android Projet Code (1255 items) and a Windows Desktop Project Code (.NET) folder (27 items) and you want to add them all to the remote repository (origin).
Now we need to add the 1,294 files to the local repository.
We can simply type the following to tell git we want it to manage all the local files.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | # Add all sub folders and files in tbhis folder. git add . # Show a local status report. simon$ git status On branch master Changes to be committed: (use "git reset HEAD <file>..." to unstage) new file: .DS_Store new file: Android Project Code/.gitignore new file: Android Project Code/.idea/.name new file: Android Project Code/.idea/compiler.xml new file: Android Project Code/.idea/copyright/profiles_settings.xml .. 1000+ orther files are listed |
Now you can merge the files top remote repository (origin).
1 | git push origin master |
Don’t forget to updates your README.md file with changes.
fyi: GitHub may take a minute or two to show changes.
1 2 3 4 5 6 7 8 | # Edt the README.md sudo nano README.md # Commit the changed file git commit README.md -m "edited README.md again" # merge chnages. push origin master |
Ignoring local files
You may want to ignore local files or folders. In your git folder type. I like to ignore all files with “secret” in the filename.
AWS recommend you ignore secret keys and logs to prevent inadvertent account access via sharing secret keys.
1 2 3 4 5 6 7 8 9 10 | # Edit your repositories ignore file sudo nano .git/info/exclude # Add These exclusions secret*.* *secret.* *secert*.* *.secret* *.*secret *.*secret* |
You can find other things to ignore here. You can setup global ignore list by reading this.
You can see I have created a few secret files and new thirdnewfile.txt (and added it to git) and committed and pushed and the secret files remain local (they will remain local while they match the ignored list).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | ls -al total 112 drwxr-xr-x 15 simon staff 510 28 Jun 15:31 . drwx------+ 53 simon staff 1802 28 Jun 14:53 .. -rw-r--r--@ 1 simon staff 12292 28 Jun 15:00 .DS_Store drwxr-xr-x 14 simon staff 476 28 Jun 15:34 .git drwxr-xr-x 15 simon staff 510 28 Jun 14:59 Android Project Code -rw-r--r-- 1 simon staff 11357 28 Jun 00:30 LICENSE -rw-r--r-- 1 simon staff 544 28 Jun 15:19 README.md drwxr-xr-x 6 simon staff 204 28 Jun 15:00 Windows Desktop Project Code -rw-r--r-- 1 simon staff 10 28 Jun 14:06 anothernewfile.txt drwxr-xr-x 5 simon staff 170 28 Jun 14:49 iOS Project Code -rw-r--r-- 1 simon staff 17 28 Jun 15:30 my.secret -rw-r--r-- 1 simon staff 10 28 Jun 01:10 newfilename.txt -rw-r--r-- 1 simon staff 26 28 Jun 15:23 secertapikey.txt -rw-r--r-- 1 simon staff 22 28 Jun 15:23 secretfile.txt -rw-r--r-- 1 simon staff 21 28 Jun 15:31 thirdnewfile.txt |
GitHub view.
Local Git View
Note the secret files.
GitHub Desktop Application
I checked out the GitHub Desktop application at https://desktop.github.com/.
The downloaded compressed file.
I extracted and copied the app to the Applications folder.
Now I logged into to the application.
Much respect to GitHub for integrating Two Factor Authentication into the login.
Unfortunately, GitHub Desktop thinks I have zero repositories (I have 9 repositories when I log in via the web using the same email address). I contacted support on Twitter but have not had a response yet?
I was able to clone the repository I made via the command line earlier or add a local repository.
I added a new text file (afileaddedwithgithubdesktop.txt) to the folder and GitHub Desktop noticed this new file straight away. I added this file to the repository.
I pushed all local changes to the origin/remote master repository.
All local changes were merged with the remote origin master (I could never do this with Atlassian’s SourceTree application).
Conclusion
Now I can sync up my projects that to GitHub repositories from the terminal and GitHub Desktop application and benefit from the features of git (backups, versioning, rollbacks, multiple machines/multiple users, auto merging etc).
Next
I will add how to pull a repository to a $5 month Digital Ocean and or AWS (Ubuntu) VM.
Read my Setting up BitBucket on an iMac and creating a project guide.
Git repositories on Raspberry Pi’s.
Related guides
Setting up a development environment on Digital Ocean 14.04
https://www.digitalocean.com/community/tutorials/how-to-install-git-on-ubuntu-14-04
Tips and Troubleshooting
GitHub has a good tips page here. I had @GitHubHelp pro-actively contact me for support on Twitter. Contact GitHub here.
How to add all local files and commit all local files.
1 | git add -A && git commit |
I hope this is helpful to someone.
P.S Don’t edit live code, use GitHub.
Agile Project Management
Read my blog post on developing software and staying on track here.
I highly recommend you follow Dmitri Iarandine at http://joinagile.com/ (Agile Coach, Trainer, Host of Lean and Mean Agile). Podcast. Author of GET HIRED as SCRUM MASTER).
Dmitri can be found here too.
Soundcloud (Lean and Mean Agile Podcast):
https://soundcloud.com/user-364782318
iTunes:
https://itunes.apple.com/au/podcast/lean-and-mean-agile-podcast/id1269551866
YouTube:
https://www.youtube.com/channel/UC00OZqfM9VmznEOTVzrQQxw
Twitter:
http://twitter.com/iarandine
Udemy:
https://www.udemy.com/agile-coaching-101/
Amazon:
https://www.amazon.com/author/iarandine
Donate and make this blog better
Ask a question or recommend an article
v1.7 added info on agile and staying on track,
How to setup and use push notifications to modern web browsers with OneSignal
Summary: A customer browses to your site and agrees to receive notifications. Days, weeks or months later you have something to tell the customer. With OneSignal you can send messages to anyone who has visited your site in the past and agreed to receive notifications. Notifications are like SMS’s but they are delivered to web browsers and are free.
Advertisement:
Sep 2016 update: This guide was written 1 year ago, it is still valid but some parts are out of date.
http://www.onesignal.com is an awesome service for sending free push notification to iOS, Android etc. Now they offer Google Chrome/Safari push notifications
to your website (on android and desktop). The guide will be updated to offer Safari setup tips as soon as Safari support is out of beta.
Check out the summary of Web Browser notifications (https://onesignal.com/chrome).
As of the 25th, you can send notifications to FireFox (daily build only).
Setting up your account (for newcomers to OneSignal)
You can use OneSignal to send push notifications to mobile apps and or just web browsers. If you already have a iOS App configured in OneSignal skip the steps below (1.x ~ 3.6) and go to step 1 (“Setting up your account (when you already have a OneSignal app for iOS“) below.
1. You will need to have a one signal account (create one)
2. Go to https://documentation.onesignal.com/docs/web-push-setup and click “Generate a GCM API Key”. (in step 3.3 add a description but not an ip). Don’t forget to save you key.
3.1. Now login to OneSignal and click “Add a new app”
3.2. Enter a “name” and click “next“.
3.3. Then click “Website Push” then click “next”
3.4 Enter your “site url” (https://www.website.com), enter your “api key” and path to an 80×80 pixel image on your web server and click “next”
3.5. Click “Website Push” and click next.
3.6 Jump to step 4 below to setup the Website SDK and upload the files to your website.
5. After you complete the steps in 4.x (below) click the”Check My Progress” to test your API key/SDK Setup and configuration.
6. Send a test push notification.
(or)
Setting up your account (when you already have a OneSignal app for iOS)
1. Open your app and click on App Settings then click Configure next to “Chrome Website (GCM)” and add your Site URL, Google Server Api Key and Icon (80×80 px png).
tip: Do not use my https://cloud.fearby.com server and yes you should really have an https site as it is more secure and boosts SEO (see this guide on buying/configuring an SSL cert)
4. One Signal Website SDK Setup
Next, we will follow the One Signal Website SDK portion of the setup (I will be using the HTTPS guide but there is an HTTP guide).
4.1. Download the latest OneSignal Website SDK code from https://github.com/one-signal/OneSignal-Website-SDK/releases
4.2. Open the Zip and upload the following files from the SetupFiles folder to the root of your website ( https://yoursite.com/manifest.json, OneSignalSDKWorker.js, OneSignalSDKUpdaterWorker.js ).
4.3. Update your manifest.json to include the “name“, “short_name” and “gcm_sender_id“.
1 2 3 4 5 6 7 8 | { "name": "<strong>Your Website Name</strong>", "short_name": "<strong>websitename</strong>", "start_url": "/", "display": "<strong>standalone", "gcm_sender_id": "123456789</strong>", "gcm_user_visible_only": true } |
4.4 Add the following into any page on your website source code that you want notifications to work on. Don’t forget to replace the app id with your one signals app id (found in your OneSignal app dashboard).
1 2 3 4 5 6 7 8 9 10 | <span class="cm-tag cm-bracket"><</span><span class="cm-tag">head</span><span class="cm-tag cm-bracket">></span> <span class="cm-tag cm-bracket"><</span><span class="cm-tag">script</span> <span class="cm-attribute">src</span>=<span class="cm-string">"https://cdn.onesignal.com/sdks/OneSignalSDK.js"</span> <span class="cm-attribute">async</span><span class="cm-tag cm-bracket">></span><span class="cm-tag cm-bracket"></</span><span class="cm-tag">script</span><span class="cm-tag cm-bracket">></span> <span class="cm-tag cm-bracket"><</span><span class="cm-tag">link</span> <span class="cm-attribute">rel</span>=<span class="cm-string">"manifest"</span> <span class="cm-attribute">href</span>=<span class="cm-string">"/manifest.json"</span><span class="cm-tag cm-bracket">></span> <span class="cm-tag cm-bracket"><</span><span class="cm-tag">script</span><span class="cm-tag cm-bracket">></span> <span class="cm-keyword">var</span> <span class="cm-variable">OneSignal</span> <span class="cm-operator">=</span> <span class="cm-variable">OneSignal</span> <span class="cm-operator">||</span> []; <span class="cm-variable">OneSignal</span>.<span class="cm-property">push</span>([<span class="cm-string">"init"</span>, {<span class="cm-property">path</span>: <span class="cm-string">"/"</span>, <span class="cm-property">appId</span>: <span class="cm-string">"<strong>b2f7f966-d8cc-11e4-bed1-df8f05be55ba</strong>"</span>, <span class="cm-property">autoRegister</span>: <span class="cm-atom">false</span>}]); <span class="cm-tag cm-bracket"></</span><span class="cm-tag">script</span><span class="cm-tag cm-bracket">></span> <span class="cm-tag cm-bracket"></</span><span class="cm-tag">head</span><span class="cm-tag cm-bracket">></span> |
4.5. Consider deleting “autoRegister: false” to have the following popup appear on your site.
Sending Notifications
1. Login to your OneSignal dashboard, open your app dashboard and click on New Push Notification.
If everything is setup you will have a Google Chrome/Safari/iOS/Android/etc Push Notifications linked up to One Signal.
The message should appear on your desktop in seconds.
When a user clicks the message they are taken to your website.
What happens when the user’s Google Chrome browser is closed and a message is sent.
The message does not deliver but as soon as the user opens Google Chrome the message appears (along with other notifications).
How Uses Opt Out
To disable notifications users can opt out.
How to add a html link based opt-in.
You can add this code to allow users to manually subscribe to notifications on your website.
1 2 3 4 5 6 7 8 9 10 11 12 13 | <body> <a href="#" id="registerForPushLink">Subscribe to Notifications</a> <script> OneSignal.push(function() { function registerForPush(event) { OneSignal.push(["registerForPushNotifications"], {modalPrompt: true}); event.preventDefault(); } document.getElementById("registerForPushLink").addEventListener('click', registerForPush, false); }); </script> </body> |
Advertisement:
Firefox Alpha Support
Firefox support is in Alpha testing mode but OneSignal inform “Firefox push notification support is in Alpha testing phase. It only works on the nightly channel of Firefox for Desktop OSes (version 44+). Firefox currently does not have a tray icon for missed notifications on Windows or Linux and clicking no a notification currently can not open a new tab yet. These 2 restrictions should be resolved once Firefox 44 is closer to release. Currently subdomain and ‘site url’ are shared with ChromeWeb. Please setup Chrome web before trying out Firefox push.”
Troubleshooting
I copied the wrong project ID from google and when I accepted the browser request to accept notifications and I could not receive notifications?, I un-checked the project in the Google chrome notifications tray icon and then re-checked it and reloaded my page 🙂
Always make sure the Home link is highlighted in the Google Developers Console and look for the project number here (and not the number in the project drop-down area).
Other
Google also allow you to send push messages directly via their servers (check out their guide: https://developers.google.com/cloud-messaging/ )
If this guide has helped please consider donating a few dollars.
Donate and make this blog better
Ask a question or recommend an article