• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • Create a VM ($25 Credit)
  • Buy a Domain
  • 1 Month free Back Blaze Backup
  • Other Deals
    • Domain Email
    • Nixstats Server Monitoring
    • ewww.io Auto WordPress Image Resizing and Acceleration
  • About
  • Links

IoT, Code, Security, Server Stuff etc

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

  • Cloud
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to buy a new domain and SSL cert from NameCheap, a Server from Digital Ocean and configure it.
    • Setting up a Vultr VM and configuring it
    • All Cloud Articles
  • Dev
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to setup pooled MySQL connections in Node JS that don’t disconnect
    • NodeJS code to handle App logins via API (using MySQL connection pools (1000 connections) and query parameters)
    • Infographic: So you have an idea for an app
    • All Development Articles
  • MySQL
    • Using the free Adminer GUI for MySQL on your website
    • All MySQL Articles
  • Perf
    • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap
    • All Performance Articles
  • Sec
    • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
    • Using OWASP ZAP GUI to scan your Applications for security issues
    • Setting up the Debian Kali Linux distro to perform penetration testing of your systems
    • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
    • PHP implementation to check a password exposure level with Troy Hunt’s pwnedpasswords API
    • Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX
    • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
    • All Security Articles
  • Server
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All Server Articles
  • Ubuntu
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Useful Linux Terminal Commands
    • All Ubuntu Articles
  • VM
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All VM Articles
  • WordPress
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
    • How to backup WordPress on a host that has CPanel
    • Moving WordPress to a new self managed server away from CPanel
    • Moving a CPanel domain with email to a self managed VPS and Gmail
    • All WordPress Articles
  • All

add

Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App

October 28, 2018 by Simon

Here is a quick guide to show you how to add two-factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA authenticator app

I have a number of guides on moving away from CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line.

Why Secure WordPress

WordPress CMS is a widely targeted CMS for hackers. View the official WordPress stats on WordPress Version/PHP and MySQL Version. View WordPress vulnerabilities here.

Read the Sucuri 2017 report on reported WordPress Hacks here (spoiler 34,371 infected websites in 2017).

Plugins exist to secure and scan WordPress. Read my blog post here on the now-retired Gravityaity Scan plugin and the awesome WordFence security plugin.

You (and hackers) can scan your site with https://wpscans.com/ or other open-source tools like wp-scan from OWASP ZAP. If you manage a WordPress site I’d recommend you install Kali Linux to scan your site.

Running a wp scan in Kali Linux is easy.

wpscan --url https://fearby.com --debug-output 2> ~/Desktop/wpscan.txt

The output from the Kali Linux wpscan tool

WPscan tool in KaiLinux

What are Hardware YubiCo YubiKeys

Read my guide here to see what YubiCo YubiKeys are and how to use them.

Yubico YubiKeys

Get the Two-Factor Plugin for WordPress Plugin

Plugin: https://en-au.wordpress.org/plugins/two-factor/

Two-Factor

Plugin Page at WordPress.org

Two Factor Auth Plugin

The source code for this plugin is available (nice): https://github.com/georgestephanis/two-factor. This plugin was updated 2 weeks ago (nice).

Downloading the Plugin

FYI: I do not allow downloading or updating of plugins in WordPress (via FTP), I prefer SSH manual downloading. FTP plugin installation and updating are not allowed on my site.

I got the latest download URL (e.g. https://downloads.wordpress.org/plugin/two-factor.zip) by copying the URL from the download button above.

I connected to my server via SSH and navigated to my WordPress plugin folder

cd /your-www-root/wp-content/plugins

I download the plugin.

[email protected]:/your-www-root/wp-content/plugins# wget https://downloads.wordpress.org/plugin/two-factor.zip
--2018-10-28 14:44:27--  https://downloads.wordpress.org/plugin/two-factor.zip
Resolving downloads.wordpress.org (downloads.wordpress.org)... 198.143.164.250
Connecting to downloads.wordpress.org (downloads.wordpress.org)|198.143.164.250|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 47882 (47K) [application/octet-stream]
Saving to: 'two-factor.zip'

two-factor.zip                             100%[=======================================================================================>]  46.76K  --.-KB/s    in 0.001s

2018-10-28 14:44:27 (37.1 MB/s) - 'two-factor.zip' saved [47882/47882]

I extracted the plugin zip file

[email protected]:/your-www-root/wp-content/plugins# unzip two-factor.zip
Archive:  two-factor.zip
   creating: two-factor/
   creating: two-factor/assets/
  inflating: two-factor/assets/banner-1544x500.png
  inflating: two-factor/assets/banner-772x250.png
  inflating: two-factor/assets/icon-128x128.png
  inflating: two-factor/assets/icon-256x256.png
  inflating: two-factor/class.two-factor-core.php
   creating: two-factor/includes/
  inflating: two-factor/includes/function.login-header.php
   creating: two-factor/includes/Google/
  inflating: two-factor/includes/Google/u2f-api.js
   creating: two-factor/includes/Yubico/
  inflating: two-factor/includes/Yubico/U2F.php
   creating: two-factor/providers/
  inflating: two-factor/providers/class.two-factor-backup-codes.php
  inflating: two-factor/providers/class.two-factor-dummy.php
  inflating: two-factor/providers/class.two-factor-email.php
  inflating: two-factor/providers/class.two-factor-fido-u2f-admin-list-table.php
  inflating: two-factor/providers/class.two-factor-fido-u2f-admin.php
  inflating: two-factor/providers/class.two-factor-fido-u2f.php
  inflating: two-factor/providers/class.two-factor-provider.php
  inflating: two-factor/providers/class.two-factor-totp.php
   creating: two-factor/providers/css/
  inflating: two-factor/providers/css/fido-u2f-admin.css
   creating: two-factor/providers/js/
  inflating: two-factor/providers/js/fido-u2f-admin-inline-edit.js
  inflating: two-factor/providers/js/fido-u2f-admin.js
  inflating: two-factor/providers/js/fido-u2f-login.js
  inflating: two-factor/readme.md
  inflating: two-factor/readme.txt
  inflating: two-factor/two-factor.php
  inflating: two-factor/user-edit.css

Enable the Plugin

Don’t forget to update the plugin in WordPress.

Enable the Plugin in WordPress

Once the plugin is enabled I can setup Two-factor authentication

Edit your Users

To setup two-factor authentication open your WordPress users screen (/wp-admin/users.php).

WordPress Users List /wp-admin/users.php

Notice the Two-Factor column

Edit your desired user to enable two-factor login options

Scroll down to Two Factor Options header, you will see a QR code that you can scan with your two-factor authentication app (e.g Google Authenticator or YubiCo Authenticator).

Enable 2FA via plugin

Always generate and save backup codes in case you lose your YubiKeys or authenticator app.

You can enable authentication methods as required.

Add the code to your Authenticator app. I will add mine to my Yubico Authenticator app that requires the insertion of a physical YubiKey. I can read my YubiKey via NFC and use my mobile phone to generate one time passwords too. Read here to learn about YubiKey 2FA (touch) devices. I have secured my Ubuntu/Debian and macOSX with these keys,

TIP: Don’t forget to save the user after editing.

Add the YubiKey 2FA (touch) to WordPress logins.

While editing a user click Register New Key under Security Keys

Add the YubiKey 2FA to WordPress

Add your primary and backup YubiKey as required (I added both of mine).

Screenshot showing two YubiKeys added to WordPress.

Enable all desired 2FA options

  • Email (OFF)
  • Time based One-Time Password (Authenticator App) (ON)
  • FIDO Universal 2nd Factor (U2F) – YubiKey Insertion and touch (ON)
  • Backup Codes (ON)

Set all desired 2FA login methods

TIP: Don’t forget to save the user after editing.

Users Table

Aim to set up every user who has access to your WordPress to use 2FA.

Mobile 2FA login

I tested logos via mobile and I was prompted to tab my YubiKey to my phone. Nice.

What happens at login?

When One Time Password is enabled as the primary authentication method I am prompted for a one-time password after entering my username and password. I then need to insert my YubiKey (or tap the YubiKey to my phone (via NFC)) to generate a one time password.

Screenshot of 2FA login prompt

When FIDO is enabled I need to insert my YubiKey and press the button.

Enter Security Key

Conclusion

I can now secure my WordPress site with 2FA protections without expensive security plugins.

I hope this guide helps someone.

More

Read more here

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.1 Added Mobile login details

v1.0 Initial post

Filed Under: 2FA, 2nd Factor, Auth, Authorization, Blog, MFA, NFC, owasp, Security, SSH, Vulnerability, Yubico, YubiKey Tagged With: 2FA, add, and, app, auth, authenticator, factor, hardware, login, or, Protection, to, two, with, wordpress, Yubico, YubiKeys

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

June 25, 2018 by Simon

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

If you have not read my previous posts I have now moved my blog from Vultr to the awesome UpCloud host (signup using this link to get $25 free credit).

Recently I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

Spoiler: UpCloud performance is great.

Upcloud Site Speed in GTMetrix

Why Upgrade

I have 1 CPU, 1 GB memory and 50GB storage and it is running well?  I have PHP child workers set up and have set up the preferred use of memory over swap file usage.

View of htop querying processes on Ubuntu

Before UpCloud, when I had 512MB ram on Vultr I had multiple NGINX crashed a day so I used a bash script and scheduled a cron job to clear memory cache when memory fell below 100MB (view the script here).  To further increase the speed of the WordPress I have configured the OS to use memory over the disk.  About once a day free memory falls below 100MB (this is not a problem as my script clears cached items automatically).

Graph of memory falling below 100MB every day

I’d like to add more memory as I am working on some things (watch this space) and I will use the extra memory. I’d prefer the server is set up now for the expected workload.

How to Upgrade

This is how I upgraded from 1xCPU/2GB Memory/50GB Storage/2TB Transferred Data to a 2 CPU/4GB Mmeory/80GB Storage/4TB Transferred Data server.

UpCloud Pricing: https://www.upcloud.com/pricing/

Pricing table form https://www.upcloud.com/pricing/

Upgrade an UpCloud VM

I shut down my existing VM. Read this guide to setup a VM.

shutdown -P

Login to the UpCloud dashboard, select your server (confirm the server has shut down) and click General Settings, choose the upgrade and click Update.

Upgrade the Server, shut it down the server and choose upgrade

I confirmed the upgrade options (2x CPU, 4096 MB Memory).

Confirm Upgrade Options

Click Update

Upgrade Applied

After 10 seconds you can start your server from the UpCloud server.

I confirmed the CPU upgrade was visible in the VM

cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 3099.978
cache size      : 16384 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 6199.95
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 3099.978
cache size      : 16384 KB
physical id     : 0
siblings        : 2
core id         : 1
cpu cores       : 2
apicid          : 1
initial apicid  : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 6199.95
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Software Tweaks Post Upgrade.

I added these settings to the top of /etc/nginx/nginx.conf to ensure the extra CPU was used.

worker_processes auto;
worker_cpu_affinity auto;

I increased PHP FPM ( /etc/php/7.2/fpm/php.ini ) to increase memory usage and child workers. I doubled child workers and max memory limit.

memory_limit = 3072M
pm.max_children = 80
pm.start_servers = 30
pm.min_spare_servers = 10
pm.max_spare_servers = 30

I restarted NGINX and PHP

nginx -t
nginx -s reload
/etc/init.d/nginx restart
service php7.2-fpm restart

I tweaked WordPress max memory limits

define( 'WP_MEMORY_LIMIT','3072M');
define( 'WP_MAX_MEMORY_LIMIT','3072M');

MySQL Tweaks: I logged into MySQL

mysql -u root -p

I ran “SHOW GLOBAL STATUS” to view stats

mysql> SHOW GLOBAL STATUS;
+-----------------------------------------------+--------------------------------------------------+
| Variable_name                                 | Value                                            |
+-----------------------------------------------+--------------------------------------------------+
| Aborted_clients                               | 0                                                |
| Aborted_connects                              | 0                                                |
| Binlog_cache_disk_use                         | 0                                                |
| Binlog_cache_use                              | 0                                                |
| Binlog_stmt_cache_disk_use                    | 0                                                |
| Binlog_stmt_cache_use                         | 0                                                |
| Bytes_received                                | 3179986                                          |
| Bytes_sent                                    | 223872114                                        |
| Com_admin_commands                            | 0                                                |
| Com_assign_to_keycache                        | 0                                                |
| Com_alter_db                                  | 0                                                |
| Com_alter_db_upgrade                          | 0                                                |
| Com_alter_event                               | 0                                                |
| Com_alter_function                            | 0                                                |
| Com_alter_instance                            | 0                                                |
| Com_alter_procedure                           | 0                                                |
| Com_alter_server                              | 0                                                |
| Com_alter_table                               | 0                                                |
| Com_alter_tablespace                          | 0                                                |
| Com_alter_user                                | 0                                                |
| Com_analyze                                   | 0                                                |
| Com_begin                                     | 0                                                |
| Com_binlog                                    | 0                                                |
| Com_call_procedure                            | 0                                                |
| Com_change_db                                 | 284                                              |
| Com_change_master                             | 0                                                |
| Com_change_repl_filter                        | 0                                                |
| Com_check                                     | 0                                                |
| Com_checksum                                  | 0                                                |
| Com_commit                                    | 0                                                |
| Com_create_db                                 | 0                                                |
| Com_create_event                              | 0                                                |
| Com_create_function                           | 0                                                |
| Com_create_index                              | 0                                                |
| Com_create_procedure                          | 0                                                |
| Com_create_server                             | 0                                                |
| Com_create_table                              | 0                                                |
| Com_create_trigger                            | 0                                                |
| Com_create_udf                                | 0                                                |
| Com_create_user                               | 0                                                |
| Com_create_view                               | 0                                                |
| Com_dealloc_sql                               | 0                                                |
| Com_delete                                    | 18                                               |
| Com_delete_multi                              | 0                                                |
| Com_do                                        | 0                                                |
| Com_drop_db                                   | 0                                                |
| Com_drop_event                                | 0                                                |
| Com_drop_function                             | 0                                                |
| Com_drop_index                                | 0                                                |
| Com_drop_procedure                            | 0                                                |
| Com_drop_server                               | 0                                                |
| Com_drop_table                                | 0                                                |
| Com_drop_trigger                              | 0                                                |
| Com_drop_user                                 | 0                                                |
| Com_drop_view                                 | 0                                                |
| Com_empty_query                               | 0                                                |
| Com_execute_sql                               | 0                                                |
| Com_explain_other                             | 0                                                |
| Com_flush                                     | 0                                                |
| Com_get_diagnostics                           | 0                                                |
| Com_grant                                     | 0                                                |
| Com_ha_close                                  | 0                                                |
| Com_ha_open                                   | 0                                                |
| Com_ha_read                                   | 0                                                |
| Com_help                                      | 0                                                |
| Com_insert                                    | 342                                              |
| Com_insert_select                             | 0                                                |
| Com_install_plugin                            | 0                                                |
| Com_kill                                      | 0                                                |
| Com_load                                      | 0                                                |
| Com_lock_tables                               | 0                                                |
| Com_optimize                                  | 0                                                |
| Com_preload_keys                              | 0                                                |
| Com_prepare_sql                               | 0                                                |
| Com_purge                                     | 0                                                |
| Com_purge_before_date                         | 0                                                |
| Com_release_savepoint                         | 0                                                |
| Com_rename_table                              | 0                                                |
| Com_rename_user                               | 0                                                |
| Com_repair                                    | 0                                                |
| Com_replace                                   | 0                                                |
| Com_replace_select                            | 0                                                |
| Com_reset                                     | 0                                                |
| Com_resignal                                  | 0                                                |
| Com_revoke                                    | 0                                                |
| Com_revoke_all                                | 0                                                |
| Com_rollback                                  | 0                                                |
| Com_rollback_to_savepoint                     | 0                                                |
| Com_savepoint                                 | 0                                                |
| Com_select                                    | 16358                                            |
| Com_set_option                                | 849                                              |
| Com_signal                                    | 0                                                |
| Com_show_binlog_events                        | 0                                                |
| Com_show_binlogs                              | 0                                                |
| Com_show_charsets                             | 0                                                |
| Com_show_collations                           | 0                                                |
| Com_show_create_db                            | 0                                                |
| Com_show_create_event                         | 0                                                |
| Com_show_create_func                          | 0                                                |
| Com_show_create_proc                          | 0                                                |
| Com_show_create_table                         | 0                                                |
| Com_show_create_trigger                       | 0                                                |
| Com_show_databases                            | 3                                                |
| Com_show_engine_logs                          | 0                                                |
| Com_show_engine_mutex                         | 0                                                |
| Com_show_engine_status                        | 0                                                |
| Com_show_events                               | 0                                                |
| Com_show_errors                               | 0                                                |
| Com_show_fields                               | 921                                              |
| Com_show_function_code                        | 0                                                |
| Com_show_function_status                      | 0                                                |
| Com_show_grants                               | 0                                                |
| Com_show_keys                                 | 1                                                |
| Com_show_master_status                        | 0                                                |
| Com_show_open_tables                          | 0                                                |
| Com_show_plugins                              | 0                                                |
| Com_show_privileges                           | 0                                                |
| Com_show_procedure_code                       | 0                                                |
| Com_show_procedure_status                     | 0                                                |
| Com_show_processlist                          | 0                                                |
| Com_show_profile                              | 0                                                |
| Com_show_profiles                             | 0                                                |
| Com_show_relaylog_events                      | 0                                                |
| Com_show_slave_hosts                          | 0                                                |
| Com_show_slave_status                         | 0                                                |
| Com_show_status                               | 6                                                |
| Com_show_storage_engines                      | 0                                                |
| Com_show_table_status                         | 0                                                |
| Com_show_tables                               | 2                                                |
| Com_show_triggers                             | 0                                                |
| Com_show_variables                            | 6                                                |
| Com_show_warnings                             | 1                                                |
| Com_show_create_user                          | 0                                                |
| Com_shutdown                                  | 0                                                |
| Com_slave_start                               | 0                                                |
| Com_slave_stop                                | 0                                                |
| Com_group_replication_start                   | 0                                                |
| Com_group_replication_stop                    | 0                                                |
| Com_stmt_execute                              | 4                                                |
| Com_stmt_close                                | 4                                                |
| Com_stmt_fetch                                | 0                                                |
| Com_stmt_prepare                              | 4                                                |
| Com_stmt_reset                                | 0                                                |
| Com_stmt_send_long_data                       | 4                                                |
| Com_truncate                                  | 2                                                |
| Com_uninstall_plugin                          | 0                                                |
| Com_unlock_tables                             | 0                                                |
| Com_update                                    | 70                                               |
| Com_update_multi                              | 0                                                |
| Com_xa_commit                                 | 0                                                |
| Com_xa_end                                    | 0                                                |
| Com_xa_prepare                                | 0                                                |
| Com_xa_recover                                | 0                                                |
| Com_xa_rollback                               | 0                                                |
| Com_xa_start                                  | 0                                                |
| Com_stmt_reprepare                            | 0                                                |
| Connection_errors_accept                      | 0                                                |
| Connection_errors_internal                    | 0                                                |
| Connection_errors_max_connections             | 0                                                |
| Connection_errors_peer_address                | 0                                                |
| Connection_errors_select                      | 0                                                |
| Connection_errors_tcpwrap                     | 0                                                |
| Connections                                   | 292                                              |
| Created_tmp_disk_tables                       | 1124                                             |
| Created_tmp_files                             | 30                                               |
| Created_tmp_tables                            | 1369                                             |
| Delayed_errors                                | 0                                                |
| Delayed_insert_threads                        | 0                                                |
| Delayed_writes                                | 0                                                |
| Flush_commands                                | 1                                                |
| Handler_commit                                | 6094                                             |
| Handler_delete                                | 33                                               |
| Handler_discover                              | 0                                                |
| Handler_external_lock                         | 38571                                            |
| Handler_mrr_init                              | 0                                                |
| Handler_prepare                               | 0                                                |
| Handler_read_first                            | 2299                                             |
| Handler_read_key                              | 134761                                           |
| Handler_read_last                             | 237                                              |
| Handler_read_next                             | 310119                                           |
| Handler_read_prev                             | 2733                                             |
| Handler_read_rnd                              | 222350                                           |
| Handler_read_rnd_next                         | 472820                                           |
| Handler_rollback                              | 0                                                |
| Handler_savepoint                             | 0                                                |
| Handler_savepoint_rollback                    | 0                                                |
| Handler_update                                | 15605                                            |
| Handler_write                                 | 17310                                            |
| Innodb_buffer_pool_dump_status                | Dumping of buffer pool not started               |
| Innodb_buffer_pool_load_status                | Buffer pool(s) load completed at 180624 23:38:01 |
| Innodb_buffer_pool_resize_status              |                                                  |
| Innodb_buffer_pool_pages_data                 | 1035                                             |
| Innodb_buffer_pool_bytes_data                 | 16957440                                         |
| Innodb_buffer_pool_pages_dirty                | 0                                                |
| Innodb_buffer_pool_bytes_dirty                | 0                                                |
| Innodb_buffer_pool_pages_flushed              | 1936                                             |
| Innodb_buffer_pool_pages_free                 | 7144                                             |
| Innodb_buffer_pool_pages_misc                 | 13                                               |
| Innodb_buffer_pool_pages_total                | 8192                                             |
| Innodb_buffer_pool_read_ahead_rnd             | 0                                                |
| Innodb_buffer_pool_read_ahead                 | 0                                                |
| Innodb_buffer_pool_read_ahead_evicted         | 0                                                |
| Innodb_buffer_pool_read_requests              | 306665                                           |
| Innodb_buffer_pool_reads                      | 950                                              |
| Innodb_buffer_pool_wait_free                  | 0                                                |
| Innodb_buffer_pool_write_requests             | 26509                                            |
| Innodb_data_fsyncs                            | 1229                                             |
| Innodb_data_pending_fsyncs                    | 0                                                |
| Innodb_data_pending_reads                     | 0                                                |
| Innodb_data_pending_writes                    | 0                                                |
| Innodb_data_read                              | 16273920                                         |
| Innodb_data_reads                             | 1078                                             |
| Innodb_data_writes                            | 2857                                             |
| Innodb_data_written                           | 53379584                                         |
| Innodb_dblwr_pages_written                    | 1275                                             |
| Innodb_dblwr_writes                           | 109                                              |
| Innodb_log_waits                              | 0                                                |
| Innodb_log_write_requests                     | 450                                              |
| Innodb_log_writes                             | 585                                              |
| Innodb_os_log_fsyncs                          | 793                                              |
| Innodb_os_log_pending_fsyncs                  | 0                                                |
| Innodb_os_log_pending_writes                  | 0                                                |
| Innodb_os_log_written                         | 664064                                           |
| Innodb_page_size                              | 16384                                            |
| Innodb_pages_created                          | 56                                               |
| Innodb_pages_read                             | 988                                              |
| Innodb_pages_written                          | 1936                                             |
| Innodb_row_lock_current_waits                 | 0                                                |
| Innodb_row_lock_time                          | 0                                                |
| Innodb_row_lock_time_avg                      | 0                                                |
| Innodb_row_lock_time_max                      | 0                                                |
| Innodb_row_lock_waits                         | 0                                                |
| Innodb_rows_deleted                           | 2                                                |
| Innodb_rows_inserted                          | 19219                                            |
| Innodb_rows_read                              | 249102                                           |
| Innodb_rows_updated                           | 77                                               |
| Innodb_num_open_files                         | 81                                               |
| Innodb_truncated_status_writes                | 0                                                |
| Innodb_available_undo_logs                    | 128                                              |
| Key_blocks_not_flushed                        | 0                                                |
| Key_blocks_unused                             | 12751                                            |
| Key_blocks_used                               | 645                                              |
| Key_read_requests                             | 321877                                           |
| Key_reads                                     | 648                                              |
| Key_write_requests                            | 196                                              |
| Key_writes                                    | 150                                              |
| Locked_connects                               | 0                                                |
| Max_execution_time_exceeded                   | 0                                                |
| Max_execution_time_set                        | 0                                                |
| Max_execution_time_set_failed                 | 0                                                |
| Max_used_connections                          | 3                                                |
| Max_used_connections_time                     | 2018-06-24 23:43:48                              |
| Not_flushed_delayed_rows                      | 0                                                |
| Ongoing_anonymous_transaction_count           | 0                                                |
| Open_files                                    | 229                                              |
| Open_streams                                  | 0                                                |
| Open_table_definitions                        | 206                                              |
| Open_tables                                   | 786                                              |
| Opened_files                                  | 502                                              |
| Opened_table_definitions                      | 208                                              |
| Opened_tables                                 | 817                                              |
| Performance_schema_accounts_lost              | 0                                                |
| Performance_schema_cond_classes_lost          | 0                                                |
| Performance_schema_cond_instances_lost        | 0                                                |
| Performance_schema_digest_lost                | 0                                                |
| Performance_schema_file_classes_lost          | 0                                                |
| Performance_schema_file_handles_lost          | 0                                                |
| Performance_schema_file_instances_lost        | 0                                                |
| Performance_schema_hosts_lost                 | 0                                                |
| Performance_schema_index_stat_lost            | 0                                                |
| Performance_schema_locker_lost                | 0                                                |
| Performance_schema_memory_classes_lost        | 0                                                |
| Performance_schema_metadata_lock_lost         | 0                                                |
| Performance_schema_mutex_classes_lost         | 0                                                |
| Performance_schema_mutex_instances_lost       | 0                                                |
| Performance_schema_nested_statement_lost      | 0                                                |
| Performance_schema_prepared_statements_lost   | 0                                                |
| Performance_schema_program_lost               | 0                                                |
| Performance_schema_rwlock_classes_lost        | 0                                                |
| Performance_schema_rwlock_instances_lost      | 0                                                |
| Performance_schema_session_connect_attrs_lost | 0                                                |
| Performance_schema_socket_classes_lost        | 0                                                |
| Performance_schema_socket_instances_lost      | 0                                                |
| Performance_schema_stage_classes_lost         | 0                                                |
| Performance_schema_statement_classes_lost     | 0                                                |
| Performance_schema_table_handles_lost         | 0                                                |
| Performance_schema_table_instances_lost       | 0                                                |
| Performance_schema_table_lock_stat_lost       | 0                                                |
| Performance_schema_thread_classes_lost        | 0                                                |
| Performance_schema_thread_instances_lost      | 0                                                |
| Performance_schema_users_lost                 | 0                                                |
| Prepared_stmt_count                           | 0                                                |
| Qcache_free_blocks                            | 1                                                |
| Qcache_free_memory                            | 16760152                                         |
| Qcache_hits                                   | 0                                                |
| Qcache_inserts                                | 0                                                |
| Qcache_lowmem_prunes                          | 0                                                |
| Qcache_not_cached                             | 16355                                            |
| Qcache_queries_in_cache                       | 0                                                |
| Qcache_total_blocks                           | 1                                                |
| Queries                                       | 19164                                            |
| Questions                                     | 19155                                            |
| Select_full_join                              | 0                                                |
| Select_full_range_join                        | 0                                                |
| Select_range                                  | 2677                                             |
| Select_range_check                            | 0                                                |
| Select_scan                                   | 2098                                             |
| Slave_open_temp_tables                        | 0                                                |
| Slow_launch_threads                           | 0                                                |
| Slow_queries                                  | 0                                                |
| Sort_merge_passes                             | 12                                               |
| Sort_range                                    | 4859                                             |
| Sort_rows                                     | 244452                                           |
| Sort_scan                                     | 854                                              |
| Ssl_accept_renegotiates                       | 0                                                |
| Ssl_accepts                                   | 0                                                |
| Ssl_callback_cache_hits                       | 0                                                |
| Ssl_cipher                                    |                                                  |
| Ssl_cipher_list                               |                                                  |
| Ssl_client_connects                           | 0                                                |
| Ssl_connect_renegotiates                      | 0                                                |
| Ssl_ctx_verify_depth                          | 0                                                |
| Ssl_ctx_verify_mode                           | 0                                                |
| Ssl_default_timeout                           | 0                                                |
| Ssl_finished_accepts                          | 0                                                |
| Ssl_finished_connects                         | 0                                                |
| Ssl_server_not_after                          |                                                  |
| Ssl_server_not_before                         |                                                  |
| Ssl_session_cache_hits                        | 0                                                |
| Ssl_session_cache_misses                      | 0                                                |
| Ssl_session_cache_mode                        | NONE                                             |
| Ssl_session_cache_overflows                   | 0                                                |
| Ssl_session_cache_size                        | 0                                                |
| Ssl_session_cache_timeouts                    | 0                                                |
| Ssl_sessions_reused                           | 0                                                |
| Ssl_used_session_cache_entries                | 0                                                |
| Ssl_verify_depth                              | 0                                                |
| Ssl_verify_mode                               | 0                                                |
| Ssl_version                                   |                                                  |
| Table_locks_immediate                         | 11962                                            |
| Table_locks_waited                            | 0                                                |
| Table_open_cache_hits                         | 19395                                            |
| Table_open_cache_misses                       | 817                                              |
| Table_open_cache_overflows                    | 12                                               |
| Tc_log_max_pages_used                         | 0                                                |
| Tc_log_page_size                              | 0                                                |
| Tc_log_page_waits                             | 0                                                |
| Threads_cached                                | 2                                                |
| Threads_connected                             | 1                                                |
| Threads_created                               | 3                                                |
| Threads_running                               | 1                                                |
| Uptime                                        | 2944                                             |
| Uptime_since_flush_status                     | 2944                                             |
+-----------------------------------------------+--------------------------------------------------+
353 rows in set (0.00 sec)

Read more on SHOW GLOBAL STATUS here. Read more on the values here.

I can see NO major errors here (possibly due to UpClouds awesome disk IO) so I won’t be making memory tweaks in MySQL. Sign Up using this link and get $25 credit free on UpCloud and see for yourself how fast they are.

Configure Ubuntu System Memory Usage

Edit /etc/sysctl.conf

Add the following to allow things to sit in ram longer

vm.vfs_cache_pressure=50

Snip from: https://www.kernel.org/doc/Documentation/sysctl/vm.txt

This percentage value controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects.

At the default value of vfs_cache_pressure=100 the kernel will attempt to reclaim dentries and inodes at a "fair" rate with respect to pagecache and swapcache reclaim.  Decreasing vfs_cache_pressure causes the kernel to prefer to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will never reclaim dentries and inodes due to memory pressure and this can easily lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100 causes the kernel to prefer to reclaim dentries and inodes.

Increasing vfs_cache_pressure significantly beyond 100 may have negative performance impact. Reclaim code needs to take various locks to find freeable directory and inode objects. With vfs_cache_pressure=1000, it will look for ten times more freeable objects than there are."

Read these pages here and here regarding setting MySQL memory.

Reboot

shutdown -r now

Resize the disk

The 2x CPU, 4GB memory plan comes with 80GB storage allowance.  My disk at present is 50GB and I will update the size soon following this guide soon.

Upgrade disk from 50gb to 80GB soon

Quick Benchmark

I used loader.io to load 500 users to access my site in 1 minute.

HTOP showing 2x busy CPU's running at 60%

The benchmark worked with no errors.

Loader.io Success with 500 concurrent users

This benchmark was performed with no Cloudflare caching. I should get Cloudflare caching working again to lower the average response time. I loaded my website manually in Google Chrome while loader.io threw 500 users at my site and it loaded very fast.

Conclusion

After a few days, I checked my memory logs and there were no low memory triggers (just normal internal memory management triggers). Ubuntu was happier.

No Low memory low triggers

This graph was taken before I set “vm.vfs_cache_pressure” so I will update this graph in a few days.

I hope this guide helps someone.

Please consider using my referral code and get $25 credit for free.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

1.0 Initial Post

Filed Under: Backup, Cloud, DB, Domain, GUI, NGINX, Performance, PHP, Scalability, Server, Ubuntu, UpCloud, Upgrade VM Tagged With: add, an, and, cpu, Disk, memory, more, on, server, Space, to, ubuntu, UpCloud, Upgrading

Using Cloudflare DNS servers to speed up the internet and add privacy on OSX

April 2, 2018 by Simon

Below is how I setup my OSX to use Cloudflare’s new DNS servers to speed up internet browsing and add privacy on OSX

Cloudflare has launched a DNS service: https://blog.cloudflare.com/announcing-1111/

DNS Performance

You can view worldwide DNS performance by viewing https://www.dnsperf.com/#!dns-providers

DNS Performance

I check the DNS at my router, I am using ISP provided DNS servers.

Review DNS

Cloudflare DNS

On April Fools 2018 Cloudflare Released a DNS server service.

Snip from here: “DNS: Internet’s Directory Nearly everything on the Internet starts with a DNS request. DNS is the Internet’s directory. Click on a link, open an app, send an email and the first thing your device does is ask the directory: Where can I find this? Unfortunately, by default, DNS is usually slow and insecure. Your ISP, and anyone else listening in on the Internet, can see every site you visit and every app you use — even if their content is encrypted. Creepily, some DNS providers sell data about your Internet activity or use it target you with ads.”

https://1.1.1.1/

Set Cloudflare Nameservers using OSX

Open the Apple System Preferences, click Network, click on your Network (Wifi or ethernet), Click Advanced then DNS and add 1.1.1.1 and 1.0.0.1

Alternatively, you can manually set your DNS servers in OSX by editing the /etc/resolv.conf, by default SX will inherit DNS settings from our router.

cat /etc/resolv.conf
#
# macOS Notice
#
# This file is not consulted for DNS hostname resolution, address
# resolution, or the DNS query routing mechanism used by most
# processes on this system.
#
# To view the DNS configuration used by this system, use:
#   scutil --dns
#
# SEE ALSO
#   dns-sd(1), scutil(8)
#
# This file is automatically generated.
#
domain home
nameserver 1.1.1.1
nameserver 1.0.0.1

Troubleshooting: Clear DNS Cache

sudo killall -HUP mDNSResponder

Debug DNS Data

scutil --dns
DNS configuration

resolver #1
  search domain[0] : home
  nameserver[0] : 1.1.1.1
  nameserver[1] : 1.0.0.1
  flags    : Request A records
  reach    : 0x00000002 (Reachable)

resolver #2
  domain   : local
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300000

resolver #3
  domain   : 254.169.in-addr.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300200

resolver #4
  domain   : 8.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300400

resolver #5
  domain   : 9.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300600

resolver #6
  domain   : a.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 300800

resolver #7
  domain   : b.e.f.ip6.arpa
  options  : mdns
  timeout  : 5
  flags    : Request A records
  reach    : 0x00000000 (Not Reachable)
  order    : 301000

DNS configuration (for scoped queries)

resolver #1
  search domain[0] : home
  nameserver[0] : 1.1.1.1
  nameserver[1] : 1.0.0.1
  if_index : 7 (en0)
  flags    : Scoped, Request A records
  reach    : 0x00000002 (Reachable)

Confirm Cloudflare DNS from the OSX Comand line

nslookup www.fearby.com
Server:		1.1.1.1
Address:	1.1.1.1#53

Non-authoritative answer:
Name:	www.fearby.com
Address: 104.27.154.69
Name:	www.fearby.com
Address: 104.27.155.69

Privacy

I am not sure if Cloudflare is any more private than using ISP DNS but I’ll happily use it.

Several people have asked me about Cloudflare’s new 1.1.1.1 privacy DNS service. To be clear: it DOES NOT stop your ISPs from collecting your browsing history. ISPs can still see the sites you’re connecting to — even if the site is over HTTPS. You will still send a hostname.

— Zack Whittaker (@zackwhittaker) April 2, 2018

Speed

I can’t tell if DNS is faster, I did ping my ISP DNS before switching and it was about the same (sub 25ms), time will tell.

Conclusion

I have used https://www.opendns.com/ before and loved the dashboards, I hope Cloudflare add dashboard options too.

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.0 Initial post

Filed Under: DNS Tagged With: add, and, Cloudflare, DNS, internet, on, OSX, privacy, servers, speed, the, to, up, Using

Add email alias to an existing GSuite email account

January 14, 2018 by Simon

This short post will show you how (and why) you should add email aliases to email accounts for previously added domains.

Here are my two posts related to setting up a new suite account and adding a second domain to an existing GSuite account. I used to have my own C Panel domain but moved all servers to self-managed servers first on Digital Ocean then AWS then Vultr.  Vultr is now my preferred hosts as they are affordable and have server locations in Australia.

Login to your G Suite with an admin account.

Login

Click Users

Click Users

I tried to add an email alias to existing 2x domains in Google  Suite (the help did not align with the screens I was seeing in my admin consoles), there was no email alias section)??

Official Google Help: https://support.google.com/domains/answer/6304345?hl=en-AU

Snip:

Every user in a G Suite account has a

primary address

 for signing in to their account and receiving mail. If a user wants another email address, you can create an

email alias

 for them.

Why use an alias? 

 If [email protected] wants a sales address to post to the company website, you can create the alias

[email protected]

. Mail sent to either address then appears in Ann's Gmail inbox.

You can add up to 30 email aliases for each user. Users continue to sign in to their G Suite account with their primary address, not an email alias.

I fired up the G Suite support

G Suite Support

I wanted to add the following email aliases of “[email protected]”, “[email protected]” and “[email protected]” to an existing email on 2 domains to support security disclosures (post soon (nope Troy Hunt beat me, read his post on security disclosure here)).

While I was waiting for the support chat I figured out the email alias thing, it turned out I needed to click the “Account” header on the user screen (I did not realise “account word was clickable).

Solution: How to add Aliases

You need to click the account heading (It’s a link apparently).

Click Account

When the Ajax content loads you can scroll down to alias section.

Alias

Tip: There is no “Apply” or “OK” button when adding aliases so just press enter.

Press enter when done

Now if you want to be able to send as the new alias you need to add the aliases in Gmail.

Alias

Now you can send via this email alias in GMail or via CLI via the primary account 🙂

sendemail -f [email protected] -t [email protected] -u "Test email via alias" -s smtp.gmail.com:587 -o tls=yes -xu [email protected] -xp YourPa$w0rdH3r3 -m "Test email via alias"

Read my blog post on sending email from the command line via G Suite.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History
v1.1 Added Troy Hunt “Data Breach Disclosures” link

v1.0 Working Copy.

Hope this helps someone.

Filed Under: Alias Tagged With: account, add, alias, an, Breach, data, Disclosures, email, existing, GSuite, to

Setting up GitHub on an iMac and creating a project

June 25, 2017 by Simon

Here is my quick guide on setting up a new GitHub repository on an iMac adding files and syncing with a  repository. Read the GitHub 101 if you are a beginner. Read my BitBucket guide here.

Why

Using git allows you to make backups, use code versioning, rollback code, use multiple development machines/have multiple users, auto merging code etc.

Github can be cheaper, more flexible and offer more features than something like Dropbox.

How

Create an account at GitHub and Login ( https://github.com/login ).

Don’t forget to set up Two Factor Authentication for extra security.

Login to GitHub (signup for a free account if need be).

Enter your Two Factor Authentication code every time you login.

Before you can read or write to your Git repository you need to set up a link between your computer (Mac) and GitHub by Setting up a Secure SSH Key with GitHub. About SSH.

If you have read any of my guides on setting up servers (e.g The quickest way to setup a scalable development IDE and web server, How to buy a new domain and SSL cert from NameCheap, a Server from Digital Ocean and configure it or Connecting to an AWS EC2 Ubuntu instance with Cloud 9 IDE as user ubuntu and root ) you will be familiar with SSH keys.

You can check your existing SSH Keys by typing the following.

ls -al ~/.ssh

Follow this GitHub guide on creating a new SSH key to use with GitHub.

I found I had to do the following in the Mac terminal to get this to work.

eval "$(ssh-agent -s)"
mkdir /gitrsatemp
cd /gitrsatemp
sudo ssh-keygen -t rsa -b 4096 -C "[email protected]"
# specify output as /gitrsatemp/githubrsa
# used a passphrase from https://www.grc.com/passwords.htm
mv /gitrsatemp/* ~/.ssh/
cd ~/.ssh/
ssh-add -K ./githubrsa

You can now add your SSH key to GitHub. 

cd ~/.ssh/ 
sudo pbcopy < ./githubrsa.pub

You can then add your public key to https://github.com/settings/keys

Add the SSH Key.

You can then test the SSH connection to GitHub.

At first, I tried this but got an error.

sudo ssh -T [email protected]
The authenticity of host 'github.com (192.30.255.113)' can't be established.
RSA key fingerprint is SHA256:key_redacted.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'github.com,192.30.255.113' (RSA) to the list of known hosts.

This is what I got when I added my SSH Key to my GitHub account. This worked for me (this updated my known_hosts file).

sudo ssh -T [email protected]
Hi SimonFearby! You've successfully authenticated, but GitHub does not provide shell access.

Tip: Don’t forget to save adding your SSH Key to GitHub

If you need help contact GitHub.

GitHub Pricing

GitHub offers free public repository pricing or $7 a month for unlimited personal repositories for personal use. Bitbucket is the Atlassian owned alternative to GitHub, they have free repositories for up to 5 users with 1TB.

Read my Setting up BitBucket on an iMac and creating a project guide.

Read on the differences between GitHub and BitBucket.

Create your first test public Repository. 

Load https://www.github.com and click Start a Project, or go to https://github.com/new

To specify a Readme file and an ignore file if need be (we can do this later). You will need to be a paid Git member to create private repositories.

When your repository is created it will be ready to upload files: https://github.com/SimonFearby/testgitproject001

Creating a local folder

Create a local folder for the repository etc.

# create a folder on your desktop
cd ~/Desktop/
mkdir testgitproject001
cd testgitproject001/

# Check the git status
git status
fatal: Not a git repository (or any of the parent directories): .git

# the folder is now ready files from the repository.

Liking the remote repository to the local folder.

First, we need to initialize git locally by typing the following.

git init
> Initialized empty Git repository in /Users/simon/Desktop/testgitproject001/.git/

Now we can add the remote “origin” repository to the local repo.

git remote add origin [email protected]:SimonFearby/testgitproject001.git

We can now see connected remote repositories by typing the following

git remote -v
origin	[email protected]:SimonFearby/testgitproject001.git (fetch)
origin	[email protected]:SimonFearby/testgitproject001.git (push)

Syncing (Pulling) the Repository

Generally, you want to pull all files from the remote repository after you create it (and the readme file).

git pull origin master

This worked 🙂

# What directory are we in
pwd
/Users/simon/Desktop/testgitproject001

# No files
ls -al
total 0
drwxr-xr-x   3 simon  staff   102 28 Jun 00:19 .
drwx------+ 53 simon  staff  1802 28 Jun 00:00 ..
drwxr-xr-x  10 simon  staff   340 28 Jun 00:21 .git

# Let's get the repo files

git pull origin master
remote: Counting objects: 4, done.
remote: Compressing objects: 100% (3/3), done.
Unpacking objects: 100% (4/4), done.
remote: Total 4 (delta 0), reused 0 (delta 0), pack-reused 0
From github.com:SimonFearby/testgitproject001
 * branch            master     -> FETCH_HEAD
 * [new branch]      master     -> origin/master

# Is the readme is there.
ls -al
total 32
drwxr-xr-x   5 simon  staff    170 28 Jun 00:30 .
drwx------+ 53 simon  staff   1802 28 Jun 00:00 ..
drwxr-xr-x  13 simon  staff    442 28 Jun 00:30 .git
-rw-r--r--   1 simon  staff  11357 28 Jun 00:30 LICENSE
-rw-r--r--   1 simon  staff     49 28 Jun 00:30 README.md

Pulling, Pushing, Branches and Forking.

You can read more about pulling, pushing, branches and forking here.

This course helped me https://www.udemy.com/learn-android/

Adding your first local file(s) to the repository

Adding all files to the new directory.

git add -A

Adding a single file:

git add newfilename.txt

Here is an example of adding a new file.

# Create a new file
touch newfilename.txt

#edit the file.
sudo nano newfilename.txt

# Tell git we want this file to be managed
git add newfilename.txt

# What is the status of this repository
git status
On branch master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

Set your default commit username (help)

git config --global user.name "Mona Lisa"

Set your default commit email address (help).

git config --global user.email "[email protected]"

Double-check config here

git config --global --edit

You will need to use vim to insert then save changes. I like the nano text editor better so I am changing the default editor.

Setting your default as nano (and not vim)

nano (and not vim)

git config --global core.editor nano

Submitting (pushing) changes

# Create a new file locally
touch newfilename.txt

# edit the local file
sudo nanonewfilename.txt

# Tell git we want to include it.
git addnewfilename.txt

# commit the file to the remote (origin) server with a commit message
git commitnewfilename.txt -m "addednewfilename.txt"

# Merge all changes with the remote (origin) server
git push origin master

The changes have been pushed (merged)

Adding another file.

This is how I added another single file locally and pushed to the remote (origin) server

# Create a new file locally
touch anothernewfile.txt

# edit the local file
sudo nano anothernewfile.txt 

# Tell git we want to include it.
git add anothernewfile.txt 

# commit the file to the remote (origin) server with a commit message
git commit anothernewfile.txt -m "added anothernewfile.txt"

# Merge all changes with the remote (origin) server
git push origin master

Now we have two new public files in a repository and locally.

README.md and markdown 

Now we need to make the README.MD file look nice and reflect the changes.

More info in markdown formatting here, here, here and here. Here is a good markdown table generator.

I made some changes (see) and pushed to the repository (origin).

# edit the file
sudo nano README.md 

# Tell git we want to commit the changes
git commit README.md -m "edited README.md"

# Merge local changes with the remote repository (origin)
git push origin master

My local README.md markdown changes

# testgitproject001
Simon's Test Git Project 001

# *Added* **two** test __text__ files (~~they are not important~~)

- [x] newfilename.txt
- [x] anothernewfilename.txt
- [ ] notmergedlocalfile.txt

## This is for my (GitHub guide on my blog](https://www.fearby.com/article/setting-up-github-on-an-imac-and-creating-a-project/)

Inline-style: 
![Simon Fearby Avatar](https://fearby.com/Avatar.jpeg "Simon Fearby Avatar")

This is what is looks like on GitHub after I commit and push

You can use pages on GitHub with markdown to create versioned documentation and help.

Commit and check the git status frequently

It is strongly advisable to frequently commit files (to a staging/not master repository) and check local it status for any issues. Commit as frequently as you remember.

# A forgotten local edit
sudo nano README.md 

# Show a gitstatus report
git status
On branch master
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

	modified:   README.md

no changes added to commit (use "git add" and/or "git commit -a")

You can push or delete changes if need be.

Adding many subfolders and many files.

Let’s say you have just added three subfolders (iOS Project Code (12 files), Android Projet Code (1255 items) and a Windows Desktop Project Code (.NET) folder (27 items) and you want to add them all to the remote repository (origin).

Now we need to add the 1,294 files to the local repository.

We can simply type the following to tell git we want it to manage all the local files.

# Add all sub folders and files in tbhis folder.
git add .

# Show a local status report.
simon$ git status
On branch master
Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

	new file:   .DS_Store
	new file:   Android Project Code/.gitignore
	new file:   Android Project Code/.idea/.name
	new file:   Android Project Code/.idea/compiler.xml
	new file:   Android Project Code/.idea/copyright/profiles_settings.xml
.. 1000+ orther files are listed

Now you can merge the files top remote repository (origin).

git push origin master

Don’t forget to updates your README.md file with changes.

fyi: GitHub may take a minute or two to show changes.

# Edt the README.md
sudo nano README.md 

# Commit the changed file
git commit README.md -m "edited README.md again"

# merge chnages.
push origin master

Ignoring local files

You may want to ignore local files or folders.  In your git folder type.  I like to ignore all files with “secret” in the filename.

AWS recommend you ignore secret keys and logs to prevent inadvertent account access via sharing secret keys.

# Edit your repositories ignore file
sudo nano .git/info/exclude

# Add These exclusions
secret*.*
*secret.*
*secert*.*
*.secret*
*.*secret
*.*secret*

You can find other things to ignore here. You can setup global ignore list by reading this.

You can see I have created a few secret files and new thirdnewfile.txt (and added it to git) and committed and pushed and the secret files remain local (they will remain local while they match the ignored list).

ls -al
total 112
drwxr-xr-x  15 simon  staff    510 28 Jun 15:31 .
drwx------+ 53 simon  staff   1802 28 Jun 14:53 ..
[email protected]  1 simon  staff  12292 28 Jun 15:00 .DS_Store
drwxr-xr-x  14 simon  staff    476 28 Jun 15:34 .git
drwxr-xr-x  15 simon  staff    510 28 Jun 14:59 Android Project Code
-rw-r--r--   1 simon  staff  11357 28 Jun 00:30 LICENSE
-rw-r--r--   1 simon  staff    544 28 Jun 15:19 README.md
drwxr-xr-x   6 simon  staff    204 28 Jun 15:00 Windows Desktop Project Code
-rw-r--r--   1 simon  staff     10 28 Jun 14:06 anothernewfile.txt
drwxr-xr-x   5 simon  staff    170 28 Jun 14:49 iOS Project Code
-rw-r--r--   1 simon  staff     17 28 Jun 15:30 my.secret
-rw-r--r--   1 simon  staff     10 28 Jun 01:10 newfilename.txt
-rw-r--r--   1 simon  staff     26 28 Jun 15:23 secertapikey.txt
-rw-r--r--   1 simon  staff     22 28 Jun 15:23 secretfile.txt
-rw-r--r--   1 simon  staff     21 28 Jun 15:31 thirdnewfile.txt

GitHub view.

Local Git View

Note the secret files.

GitHub Desktop Application

I checked out the GitHub Desktop application at https://desktop.github.com/.

The downloaded compressed file.

I extracted and copied the app to the Applications folder.

Now I logged into to the application.

Much respect to GitHub for integrating Two Factor Authentication into the login.

Unfortunately, GitHub Desktop thinks I have zero repositories (I have 9 repositories when I log in via the web using the same email address). I contacted support on Twitter but have not had a response yet?

I was able to clone the repository I made via the command line earlier or add a local repository.

I added a new text file (afileaddedwithgithubdesktop.txt) to the folder and GitHub Desktop noticed this new file straight away. I added this file to the repository.

I pushed all local changes to the origin/remote master repository.

All local changes were merged with the remote origin master (I could never do this with Atlassian’s SourceTree application).

Conclusion

Now I can sync up my projects that to GitHub repositories from the terminal and GitHub Desktop application and benefit from the features of git (backups, versioning, rollbacks, multiple machines/multiple users, auto merging etc).

Next

I will add how to pull a repository to a $5 month Digital Ocean and or AWS (Ubuntu) VM.

Read my Setting up BitBucket on an iMac and creating a project guide.

Git repositories on Raspberry Pi’s.

Related guides

Setting up a development environment on Digital Ocean 14.04

https://www.digitalocean.com/community/tutorials/how-to-install-git-on-ubuntu-14-04

Tips and Troubleshooting

GitHub has a good tips page here.  I had @GitHubHelp pro-actively contact me for support on Twitter. Contact GitHub here.

How to add all local files and commit all local files.

git add -A && git commit

I hope this is helpful to someone.

20 essential git tips. 

P.S Don’t edit live code, use GitHub.

Agile Project Management

Read my blog post on developing software and staying on track here.

I highly recommend you follow Dmitri Iarandine at http://joinagile.com/ (Agile Coach, Trainer, Host of Lean and Mean Agile). Podcast. Author of GET HIRED as SCRUM MASTER).

Dmitri can be found here too.

Soundcloud (Lean and Mean Agile Podcast):
https://soundcloud.com/user-364782318

iTunes:
https://itunes.apple.com/au/podcast/lean-and-mean-agile-podcast/id1269551866

YouTube:
https://www.youtube.com/channel/UC00OZqfM9VmznEOTVzrQQxw

Twitter:
http://twitter.com/iarandine

Udemy:
https://www.udemy.com/agile-coaching-101/

Amazon:
https://www.amazon.com/author/iarandine

Donate and make this blog better




Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.7 added info on agile and staying on track,

Filed Under: Git, GitHub Tagged With: add, code, git, merge, pull, push, rfep[ository, rollback, versions

Primary Sidebar

Poll

What would you like to see more posts about?
Results

Support this Blog

Create your own server today (support me by using these links

Create your own server on UpCloud here ($25 free credit).

Create your own server on Vultr here.

Create your own server on Digital Ocean here ($10 free credit).

Remember you can install the Runcloud server management dashboard here if you need DevOps help.

Advertisement:

Tags

2FA (9) Advice (17) Analytics (9) App (9) Apple (10) AWS (9) Backup (21) Business (8) CDN (8) Cloud (49) Cloudflare (8) Code (8) Development (26) Digital Ocean (13) DNS (11) Domain (27) Firewall (12) Git (7) Hosting (18) IoT (9) LetsEncrypt (7) Linux (21) Marketing (11) MySQL (24) NGINX (11) NodeJS (11) OS (10) Performance (6) PHP (13) Scalability (12) Scalable (14) Security (45) SEO (7) Server (26) Software (7) SSH (7) ssl (17) Tech Advice (9) Ubuntu (39) Uncategorized (23) UpCloud (12) VM (45) Vultr (24) Website (14) Wordpress (25)

Disclaimer

Terms And Conditions Of Use All content provided on this "www.fearby.com" blog is for informational purposes only. Views are his own and not his employers. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. Never make changes to a live site without backing it up first.

Advertisement:

Footer

Popular

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Add Google AdWords to your WordPress blog

Security

  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • Setting up DNSSEC on a Namecheap domain hosted on UpCloud using CloudFlare
  • Set up Feature-Policy, Referrer-Policy and Content Security Policy headers in Nginx
  • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
  • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
  • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
  • Beyond SSL with Content Security Policy, Public Key Pinning etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Run an Ubuntu VM system audit with Lynis
  • Securing Ubuntu in the cloud
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider

Code

  • How to code PHP on your localhost and deploy to the cloud via SFTP with PHPStorm by Jet Brains
  • Useful Java FX Code I use in a project using IntelliJ IDEA and jdk1.8.0_161.jdk
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider
  • How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic
  • Installing Android Studio 3 and creating your first Kotlin Android App
  • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
  • How to use Sublime Text editor locally to edit code files on a remote server via SSH
  • Creating your first Java FX app and using the Gluon Scene Builder in the IntelliJ IDEA IDE
  • Deploying nodejs apps in the background and monitoring them with PM2 from keymetrics.io

Tech

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Is OSX Mojave on a 2014 MacBook Pro slower or faster than High Sierra
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • The case of the overheating Mac Book Pro and Occam’s Razor
  • Useful Linux Terminal Commands
  • Useful OSX Terminal Commands
  • Useful Linux Terminal Commands
  • What is the difference between 2D, 3D, 360 Video, AR, AR2D, AR3D, MR, VR and HR?
  • Application scalability on a budget (my journey)
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.

Wordpress

  • Replacing Google Analytics with Piwik/Matomo for a locally hosted privacy focused open source analytics solution
  • Setting web push notifications in WordPress with OneSignal
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Wordfence Security Plugin for WordPress
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
  • Moving WordPress to a new self managed server away from CPanel
  • Moving WordPress to a new self managed server away from CPanel

General

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Using the WinSCP Client on Windows to transfer files to and from a Linux server over SFTP
  • Connecting to a server via SSH with Putty
  • Setting web push notifications in WordPress with OneSignal
  • Infographic: So you have an idea for an app
  • Restoring lost files on a Windows FAT, FAT32, NTFS or Linux EXT, Linux XFS volume with iRecover from diydatarecovery.nl
  • Building faster web apps with google tools and exceed user expectations
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..

Copyright © 2023 · News Pro on Genesis Framework · WordPress · Log in

Some ads on this site use cookies. You can opt-out if of local analytics tracking by scrolling to the bottom of the front page or any article and clicking "You are not opted out. Click here to opt out.". Accept Reject Read More
GDPR, Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT