Advice

Setup a Certification Authority Authorization (CAA) DNS record(s) to prevent https cert issue/misuse

March 18, 2019 by Simon

On February 22nd 2017 CAA’s that issue https certificates are required to check what CAA’s are allowed to issue HTTP’s certificates for a website. To limit who can create HTTP’s certificates for your site all you need to do is specify a number of DNS records.

DNSSEC

Before adding DNS CAA records ensure you have enabled DNSSEC for extra security, this is not needed to setup CAA records but it’s a good idea.

DNSSEC Explained

Read my post here on setting up DNSSEC with Cloudflare here.

Namecheap allows you do set DNSSEC with 1 click (making the above guide not required unless you use Cloudflare).

Testing DNSSEC

First, test DNSSEC on your website here: https://dnssec-analyzer.verisignlabs.com/ (I already have DNSSEC enabled)

I use Namecheap for buying domains and HTTP’s certs (you can buy a new domain here). Namecheap allow you to easily enable DNSSEC and CAA DNS records.

Read Namecheap’s CAA guide here.

Scott Helme tagged a great write up on CAA here.

CAA is probably the best bang for buck you’re going to get! https://t.co/pvThaQ8qFl
— Scott Helme (@Scott_Helme) March 14, 2019

Testing CAA (on your website)

Go to https://dev.ssllabs.com/ssltest / and scan your website

https://dev.ssllabs.com screenshot showing a domain input box

You will see if CAA is enabled after the https test is complete (scroll past the rating)

In my case CAA records were not detected.

Adding DNS CAA records at Namecheap

I logged into Namecheap, clicked Manage domain and clicked the Advanced DNS tab

Screenshot showing Namecheap Advanced DNS screen.

I click Add New Record (DNS), then I selected CAA

Screenshot of add NDS CAA record at Namecheap

Here are records for my main domain (allowing Comodo/Sectigo HTTP’s certificates only)

Type, Host, Value, TTL

CAA Record @ 0 issue "comodoca.com" Automatic
CAA Record @ 0 issue "comodo.com" Automatic
CAA Record @ 0 issue "usertrust.com" Automatic
CAA Record @ 0 issue "trust-provider.com" Automatic
CAA Record @ 0 issue "sectigo.com" Automatic

Here is my record allowing a sub domain (allowing Lets Encrypt HTTP’s certificates only)

Type, Host, Value, TTL

CAA Record audit.fearby.com 0 issue "letsencrypt.org" Automatic

It is also possible to setup email alerts of CAA violations where CAA’s support it. I setup a [email protected] email alias.

Type, Host, Value, TTL

CAA Record audit.fearby.com 0 iodef "mailto:[email protected]" Automatic
CAA [email protected] 0 iodef "mailto:[email protected]" Automatic

Image of my final Namecheap DNS config.

Screenshot os Namecheap DNS entries (table below)

Test CAA Records

I visited https://dev.ssllabs.com/ssltest / and performed a final scan.

Pass 🙂

I do have real time remote server monitoring reporting on https presence and uptime, read the post here.

Plug(s)

Warning

I had an issue where I failed to update my DNS (and define a CAA record) for the sub domain used for Nixstat reporting. I was receiving this error.

dev.ssllabs.com was reporting the cert expired?

The awesome chat support (Vincent) over at Nixstats found out it was because I did not have CAA record for the sub domain allowing “letsencrypt.org” to generate certs.

Created CAA record for status.feabry.com (CAA 0 issue "letsencrypt.org"

If you manually renew a Lets Encrypt cert with the following command without a CAA record you will see an error

> certbot -q renew

Error Output

Attempting to renew cert (subdomain.fearby.com) from /etc/letsencrypt/renewal/
subdomain.fearby.com.conf produced an unexpected error: Failed authorization procedure.
subdomain.fearby.com (http-01): urn:acme:error:caa :: CAA record for
subdomain.fearby.com prevents issuance. Skipping.
All renewal attempts failed.

DNS additions and changes take a while to propagate so monitor Whats My DNS for change status

https://www.whatsmydns.net/#CAA/status.fearby.com

Thanks for reading.

For simplicity I have removed all sub domain CAA settings for records and only set global ones

Revision History

v1.2 Troubleshooting

v1.1 Plugs

v1.0 initial Post

I am moving away from Apple hardware

February 3, 2019 by Simon

My Late 2012 Mac Book Pro Retina laptop is all but dead, it has many dead pixels and because of the poor cooling and is NOT a joy to use anymore. It does not “JUST WORK” and personally, I do not think “thinner” laptops can handle Australian summers as its hardware cooling it inadequate above 40c air temperatures.

My laptop processor would spend more time thermal throttling (at 104c) in Web Browsers and text editors that at normal speeds. Opening up productivity apps like Photoshop or Premiere Pro would send the laptop into meltdown.

Frequent high temps were common.

Attempted Fixes

Warning Disclaimer: My laptop is out of warranty and I know my way around the inside of computer hardware without zapping it. Do not attempt to open your laptop unless you know what you are doing, have backed up your data and are prepared to brick your computer.

I removed dust from inside the laptop.
I tried to only use the laptop refrigerative air conditioning
I replaced the thermal paste on the CPU and GPU (3 times)
I reinstalled OSX Mojave and reset the SMC and PRAM multiple times.
I ran the fans at 100% (see post here), The fans were operating at full capacity and were not broken.

The stock thermal paste was crusty after 5 years. The plastic CPU/GPU cover was visibly cooked.

I ordered some new Thermal grizzly thermal paste, I had some older silicone paste on hand just in case.

After many reapplications of the Thermal Grizzly, the older silicone paste seemed to work the better???

After a few months, all of the fixes above did not seem to work. OSX Mojave would spin up the CPU and GPU into a frenzy overloading the single heat pipe within minutes.

Time to try some more drastic cooling modifications?

I tried improving the efficiency of the single (copper) heat pipe that is shared between an Intel i7 2.6 GHz and an Nvidia Video Card by removing the black paint by stripping the paint with acetone.

I manually removed paint from in between the heat sink fins with a LED to reveal the metal.

I reinstalled the heat pipe with high hopes? That looks nice 🙂

I removed the old thermal paste and added new paste. First I tried Thermal Grizzly Cryonaut. I re-applied the paste three separate times as each application was not that much better than the old crusty stock paste from Apple. Did I have a bad batch of Thermal Grizzly?, It seemed thick and not very viscous. I ended up using an old tube of silicone paste (the white stuff) as my Arctic Silver was too old to try and I did not want to order more.

With the silicone paste applied and the paint removed temperatures were about 15c lower at max, I still had frequent thermal throttling but at least I had a reserve buffer.

This was all before the Aussie Heatwaves and high temperatures soon returned.

Is there still room for improvement?

How heat pipes work

Heat pipes have an evaporating (hot part) and condensing zone (cool part) on the heat pipe. I noticed Apple’s “stock” condensing fins were small, would improving this zone help?. Time to improve the condensers zones by adding larger copper heat sinks to the bare side of the heat pipe.

I purchased a few copper Xeon/Sun server sized heat sinks and thermal epoxied them to the condensing end of the heat pipe. Yes, they would protrude out the bottom of the case but #Meh. I can fix that by extending the base of the laptop down and making it thicker (old school style).

The server heat sinks arrived

I cut the heat sinks in half.

I packed the fins with paper before cutting to ensure the cut did not damage the fins.

After cutting, I wiped the copper heat sinks with vinegar to restore the surface to a nice copper shine.

I tested the heat sink idea with silicone paste first

Temps were 25c lower, Now it’s time to use Arctic Silver Thermal Epoxy

I applied the Thermal Epoxy to the heat pipe (I temporary had foil strips above the fans so I did not block them while the epoxy dried.

I then stuck the heat sink’s to the heat pipe (with Arctic Silver Thermal Epoxy).

I toyed with a clear case but decided against it for static electricity and stability reasons.

I purchased a second Mac Book base for so I could cut holes for the heat sinks to protrude and use the original base to hide the modification.

I made a 30 mm base wall so I could use it as a wall between the laptop base and the new 30mm lower base.

I added some 5-volt and 12 -volt fans inside the new extended 30mm base.

Finished Product

A normal looking Mac Book except for the 30mm lower base and internal 5V or 12V fans.

External power plugs on the left side, I will add lights at a later stage.

Are the temps lower?

Videos

Video: Mac Book Pro cooling mod, I can now watch 1080p videos without maxing the CPU

Video: Mac Book Pro cooling mod with external powered 5v or 12v fans

Conclusion

50c lower temps are nicer at idle but in Premiere Pro (exporting video) the laptop was still thermal throttling like mad and temps were terrible (100+). Lets not get started when I start some development VM’s

Conclusion 2 weeks later

This is still not a joy to use. I don’t think I have the right to expect a 5-year-old laptop to keep up running a CPU/GPU intensive OS and applications.

Time to buy a new computer, Apple still makes thin and overheating laptops by the looks of it?

Maybe I need to buy a fridge to stick a computer in a fridge to use these days?

YouTube users indicate Apple has a problem with heat.

What computer do I get next?

Not an Apple made one. I will be moving back to Windows for local development and Linux on servers

Dell Alienware has many heat pipes.

Acer Predator 500

I read a few reviews (e.g this one from Ultra book reviews) and Acer have good cooling.

MSI GT Series laptops look the best if cooling is important.

Or should I build a custom desktop with way more cores

CPU: Threadripper 2950X 16C 32T

SSD: M.2 SSD: Samsung 970 PRO 512GB
MOBO: Asus Zenith Extreme
Power: Corsair RM1000x 1000W
MEM: Quad 3600 Mhz
GPU: AMD Radeon VII Navi 3980

Thanks for reading.

v1.3 Added videos

v1.2 Updated alt tag descriptions

v1.1 Added “I will be moving back to Windows for local development and Linux on servers”

1.0 Initial Draft

Using Platforma Web Wireframe Kit to build a website (prototype)

November 24, 2017 by Simon

I have blogged before about building a server for users to install WordPress, optimizing images in WordPress, deploying WordPress via CLI, moving WordPress, speeding up WordPress and securing WordPress but what do you do if you want a non-WordPress site without the support hassles?

Recently I gave the https://platforma.ws iOS prototyping library extensions a test. I was delighted to find they had a Web Wireframe Kit (generation suite) for prototyping and exporting working websites. You can try the free version or buy a licence here.

Creating a Website with Platforma Web Wireframe Kit

Goto https://platforma.ws and click HTML Generator (or click here)

You will be presented with an empty website ready for your attention.

Adding Website Elements

It’s as simple as clicking a purple add button.

This reveals a number of HTML templates samples that you can drag and drop to your website design.

You can then choose a category (e.g “Header”) and see the elements of the available sample.

Simply drag and drop the elements out into your design.

Now, Let’s click the purple Add (in the top left) button and add a sample Header section, sample Contents, sample Slider, sample Body, sample first Call to Action section, sample Pricing Table, sample second Call to Action section, sample Footer section.

30 seconds later and I have generated designed a site ready to edit the exported HTML.

Exporting Your Site from Platforma

Click on the Export button (in the top right).

I was greeted with the following export screen, this page explains the difference in export options: http://app.platforma.ws/docs/

I don’t need “node.js” or “gulp” “Advanced Version” (PUG + STYLUS) so I’ll choose “Simple Version” (HTML + CSS + JS).

You will need to enter a licence key to continue the export.

The website export download came down just fine.

Code

The code looks ok, I did notice that images were missing alt tags so I added those in.

Any Errors in the Code (in Chrome)?

Nope, Chrome loads the code with no errors.

Testing Online

How about HTML5 and WCAG 2.0 AA

I uploaded the zip file to my server (using the scp command), I could have used SFTP.

scp /local/folder/local-file.zip [email protected]:/www/destination-folder/

I unzipped the site with

sudo apt-get install unzip
unzip filename.zip

The site loads just fine in a web browser

Accessibility

I used https://achecker.ca/checker/index.php to test the site with WCAG 2.0 AA, the only remaining issues I found were in relation to the multiple H1, H2 etc tags (this can be fixed by moving the H CSS code to custom classes and removing H1, H2 etc tags altogether (and reference the custom class matching the H* tags)).

fyi: The potential WCAG problems that were being alerted were in relation to…

My alt tags were potentially short
Potential Colour warnings
Potential Contrast warnings
Missing a “Skip to content” block
Reporting of placeholder graphics and alt tags (a checker is smart)
etc

I tested the sites HTML compliance with https://validator.w3.org/, the code passed with flying colours.

Customizing

I could not find a way to edit the elements in the http://app.platforma.ws/# like the Platforma iOS Adobe XD Kit but you can quickly edit in your HTML after exporting (using your editor of choice like Dreamweaver, Sublime or Notepad).

Conclusion

Platforma Web Wireframe Kit is an essential tool for anyone wanting to build quick web prototype (or even live sites) website for themselves, clients etc. I am very impressed with the code created.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.1 Added more, fixed a bit.

etc

Building faster web apps with google tools and exceed user expectations

November 4, 2017 by Simon

@jawache pointed me in the direction of twitter user @addyosmani and a talk at Chrome Dev Summit 2017 called “Fast By Default: Modern Loading Best Practices

Recently I blogged about speeding up my serves (by moving them closer to me and visitors) and Sped up WordPress by moving away from CPanel tp a self-managed server.

Update 2018: For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

RAIL: Measure Performance with the RAIL Model (Fast Integrated, Reliable and Engaging)

https://developers.google.com/web/fundamentals/

Focus on the user
Initial Response under 100ms
Animate, produce a frame in under 10ms
Idle maximize idle time
Deliver all content in under 1000ms

Replaced with: User happiness metrics

First Paint
First content Paint
First Meaningful Paint
Time to interactive

Potential Impacts

Network ( See my scalability on a journey on a budget blog post)
CPU throttling
Client Heavy Frameworks (JavaScript is heavy parsing)
Shared hosts (Moving away from a shared host to AWS from CPanel or other shared hosts can be a good thing for performance. )
Network Latency
Size (number and file size)
HTTP ASync Downloads Limit (2)
Device Speed
Parsing JavaScript.
3r party libraries
Caching (web server (NGINX caching https://www.nginx.com/blog/nginx-caching-guide/ ) or local browser cache)
User Limitations
etc

Do’s

Less JavaScript
More Caching
Lazy Loading ( I use the BJ Lazy Load plugin in WordPress )
Develop on Slow Devices/Links
CDN
DNS
Minify Code ( fyi )
Optimize Images ( fyi )
Respect Visitor Data Plans/Devices (offer low data version)

Use online tools (Browser dev audit tools) to measure performance metrics.

Consider in-memory caching solutions

Nginx Caching
OpCache if you are using PHP.
Node-cache – In memory caching.
Redis – In memory caching.

Use the following third-party tools while developing.

These are highly recommended tools.

Siege Command Line Benchmarking tool (Read guide here)

https://calibreapp.com/

https://speedcurve.com/

https://github.com/siddharthkp/bundlesize

http://loader.io/ – External Benchmarker

State of the Web:

Use http://beta.httparchive.org/ to see the average website stats and trends, is your site average or abusing the code type/size you push?

Sample httparchive.org Reports:

Average TCP Connection Per Page

The average percent of HTTPS requests (now over 50%)

Time in seconds to first meaningful paint

Average time to first interactive paint

Average time to first on load (finished)

Netcraft: October 2017 Web Server Survey:

https://news.netcraft.com/archives/2017/10/26/october-2017-web-server-survey-13.html

Web Server Technology Share

(Image credit: netcraft.com)

Active Web Sites v Hostnames

(Image credit: netcraft.com)

State of Desktops (Gamers Survey)

Steam conducts a monthly survey to collect data about what kinds of computer hardware and software our customers are using. Participation in the survey is optional, and anonymous via the steam app.

This gives a rough (biased) breakdown of high-end machines (gamers) but does reveal valid stats on installed memory and product share.

http://store.steampowered.com/hwsurvey

Don’t forget to track the visitors (location, time, devices, speed, gender etc.) to your site with Google Analytics https://support.google.com/sites/answer/97459?hl=en

Google Progressive Web APP LightHouse Site Audit (Chrome Dev Tools)

You can now run Audits in the Chrome DevTools. To run a report (Read More: https://developers.google.com/web/tools/lighthouse/#devtools )

Click Perform and Audit (in the Chrome Developer Tools) on the site you have loaded.

Audit in Progress

My Report

FYI: Report scores will depend on the time of day and the speed of your computer, A 55 score maybe 65 or 45 the following day.

Read the Progressive Web App Check List to help understand how to boost your score.

https://developers.google.com/web/progressive-web-apps/checklist

Google Mobile-Friendly Check List

Also, run the mobile-friendly tesh over your site here.

https://search.google.com/test/mobile-friendly

Google Page Speed Insights

Goto PageSpeed Insights and scan your site.

Page Speed Insights Report Results

FYI: My site is a WordPress site and I speed up the sight by.

Moving away from a shared CPanel server
Optimized Images in WordPress with the EWWW plugin
Made speed improvements in WordPress
Wordfence Security Plugin for WordPress buy also set up a WordFence firewall (slowing things down)
etc

The next thing I want to try is (Sponsors needed).

Setting up self-managed Sub Domain CDN’s (https://img01cdn.fearby.com, https://img02cdn.feearby.com etc). – See Vultr and Digital Ocean Ubuntu Server setup guides.
Third Party CDN’s.
DNS optimizations.
Setup NGINX Caching.

Google User Experience Report (Advanced):

The Google User Experience Report (SQL like query and reporting system) is a public dataset of key user experience metrics for top origins on the web. All performance data included in the report is from real-world conditions, aggregated from Chrome users who have opted-in to syncing their browsing history and have usage statistic reporting enabled ( https://www.google.com/chrome/browser/privacy/whitepaper.html#usagestats ) .(end snip from https://blog.chromium.org/2017/10/introducing-chrome-user-experience-report.html )

Getting started with Chrome User Experience Report Tool: https://developers.google.com/web/tools/chrome-user-experience-report/getting-started
Chrome User Experience Report Tool will allow you to query datasets on page load times.

Why

Better able to compete and deliver measurable advantages to your customers.

Conclusion

Know data – MOM (Measure, Optimise and Monitor).
Improve over time or before launch.
Set budgets on file sizes and page times.
Develop on slow devices/links.
Consider using PageSpeed Module on your server https://developers.google.com/speed/pagespeed/module/
Consider using the RPRL Pattern (https://developers.google.com/web/fundamentals/performance/prpl-pattern/ ) to split and cache code to speed up the time to interactivity.
Scale up or out?
Enabling web server caching (NGINX: https://www.nginx.com/blog/nginx-caching-guide/

Still Reading?

View more Google Dev Summit 2017 talks: https://www.youtube.com/playlist?list=PLNYkxOF6rcICUD5nBfRdAR6Fveosnqa5m

Also read: Can You Afford It?: Real-world Web Performance Budgets: https://infrequently.org/2017/10/can-you-afford-it-real-world-web-performance-budgets/

Still Reading:

Check out Googles HTTPS Transparency Report https://transparencyreport.google.com/https/overview

Read the introduction blog at Chromium Blog https://blog.chromium.org/2017/10/introducing-chrome-user-experience-report.html

Bookmark https://blog.chromium.org/

Update Jan 2018

Google begins mobile-first indexing of sites to improve mobile search results

What Crawl Budget Means for Googlebot

More to come.

Donate and make this blog better

Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.2 Jan 2018 additions on Google’s mobile first approach and increasing crawl rates.

v1.1 fixed typos

Infographic: So you have an idea for an app

October 31, 2017 by Simon

I created this graphic as I was asked by multiple people how to develop an app. This does not include tips on coding but many people with the non-technical prerequisites to building an app.

I hope this graphic helps someone (It’s my first infographic/decision flow image, feedback welcome).

So You Have an Idea For An App: Graphic

Click for a larger version.

Standalone Image URL’s

v1.3 (22nd November 2017)
  https://fearby.com/wp-content/uploads/2017/10/Infographic-So-you-have-an-idea-for-an-app-v1-3.jpg
v1.2 (4th Nov 2017, Added requirements and MoSCoW): 
  https://fearby.com/wp-content/uploads/2017/10/Infographic-So-you-have-an-idea-for-an-app-v1-2.jpg
v1.1 (1st Nov 2017, Fixed Typos): 
  https://fearby.com/wp-content/uploads/2017/10/Infographic-So-you-have-an-idea-for-an-app-v1-1.jpg

todo: Things to add Issues to fix in 1.4:
 - Add user personas and Epic, Story and Task stages.
 - How to capture good stories (and validated ideas (landing pages/interviews/problems/value/painpoints)

Define the problem(s) (pain points)

Before you start coding, do list your app requirements (problem’s to solve (pain points)).

Atlassian JIRA or Trello can help with this. I personally use (and like) Atlaz.io (now Hygger), I reviewed the BETA here).

Using Trello lists are also a simple way to capture tasks/ideas.

More on these Read more here also read my Atlaz.io BETA Preview here.

Nothing beats pen and paper too.

Moscow Prioritization

Must-Have Should-Have, Could-Have and Won’t-have are buckets you should sort ideas into. If you have trouble moving items away from Must to Should, Could or Won’t then assign a fictitious monetary value to spend on each item and that will help you decide what is more important.

Read this MoSCoW Method article at Wikipedia: https://en.wikipedia.org/wiki/MoSCoW_method

Managing MoSCoW tasks on paper is OK if you do not want to use planning software.

More

Read my guide on how to prototype apps with Adobe XD guide here. You can also Prototype a Web app with Platforma (review here).

Read my post on how to develop software and stay on track.

Research

Do research your idea for market fit/need, competition, complexity, legal and validate ideas early. It’s best to find out early that Google will quote $60,000+ TAX a year to allow you to use Google map’s in your app early, then you can use https://www.mapbox.com for $499 a year.

Do you have competition?

Some people say “don’t develop an app that already exists”. Why would you develop a new Uber app? Henry Ford did make a new transportation mode when people were happy with horses, other car manufacturers like Tesla are moving in on the space so don’t be discouraged.

Landing Page

A landing page with a signup form (Newsletter and Register Interest) form is a good way to validate ideas and get feedback early (I would suggest you use a free Mainchimp signup form, a generated website with Platforma on a $5/m server for quick results). There is no point coding and launching to crickets.

Do you have an app Prototype or Mock-Up?

This is very important and easy step. Programs like Adobe XD CC (read my guide here) and Balsamiq can help you prototype an app, Platforma can help you prototype web apps.

Have you validated your idea (app) with end-users?

If you don’t do this you are mad. Watch this video to see lessons learned from Trades Cloud.

Is this app idea a hobby (passion)?

This can help you limit costs and expectations. Cheap serves exist (read here and here).

Do you have time to develop/manage this?

Developing and managing an app and planning (paying for) development cycle can be time-consuming and mentally draining.

Can you code?

Do you need to hire developers or learn to code? Blog post coming soon on how to hire coders.

Do you have funds?

Having funds on hand to set up and build an app is very important.

Do you want to hide developers (or get Venture Capital)?

This can help you get moving but you will have to give away a slice of the profits and or IP, managing mentors and VC’s can be tiresome.

Have you set failure criteria (post-mortem)?

Read this page on lessons learned from over 200 startup failures, save your favourites. Having realistic goals and limits is a wise idea, do stop when you reach preset limits.

Do you have a business case?

There is plenty of business case generator template’s, you will want to document some of the following.

What is your apps Purpose – App X will be..
What is your Mission Statement – App X will..
Who are your Target Customers – Retail..
Who are the Early Adopters – Retail..
What Problems does your app solve – App X will..
What Milestones will your app go through – iOS, Android, Apple TV, Web etc..
What Existing solutions exist – App: A, B and C..
How does your app Solve your customer’s problems (pain points) – App X will..
How will your app Find customers – Word of Mouth, Referrals, Advertisements?
What is your Revenue model – Sales, Ad’s, Subscriptions?
What is your apps Goal statement – App X will hit X users in X?
What are your apps Failure points – If app X does not reach X or monthly costs reach Y….
What is your Marketing message – App X will..
What is your apps Metrics – iOS, Android, Apple TV apps..
What is your Unfair Advantage – Why will you succeed over others?

Are you using a project management methodology?

Proven Methodology can help you develop software and stay on track, software like Atlaz, JIRA or Trello are highly recommended tools. Capturing ideas and processing feedback in tools is very important.

Before you code (or hire coders) use source code versioning software like GitHub and Bitbucket (guides here and here). You want to retain the code and insist on owning it.

Product Goal

Simon Sinek has a good video on companies (or Products) being in a finite or infinite game.

Are you in full control of your development stack?

If you are not a developer you may not care if you are in control, but you will if there are issues with hired developers or issues with service providers. I moved from CPanel to self-managed servers, moved from IBM Cloudant to Digital Ocean to AWS then Vultr servers where I can have full control or scalability, features, security and costs.

Can you forecast the costs?

Lowering cost and boosting performance is important and having spare money is a good thing.

I read recently that Telsla is burning through $6,000 a minute and is forecast to need something like 2 billion dollars in the next 2 years. Software as Service platforms will drain your budget quick (they do take on some risk and maintenance tasks), is this worth it?

Mark Fedin (CEO and Co-founder at Atlaz) has a great post on the topic of viability Stop Dabbling At Startups .

Are you using the right tech?

Don’t be afraid of changing tech along the way, you may start with MySQL and move to MongoDB, Redis, Oracle ot MSSQL database servers etc.

Do you have systems to capture customer feedback?

Self-explanatory, you are solving customer problems, right? You will pivot in the first year (trust me).

What is your revenue/sales model?

If you don’t know how to make money then don’t make an app (apps are expensive to code and maintain).

Are you prioritizing task?

I have blogged about this before, do use the tools to stay on track.

Funny Bit

Donate and make this blog better

Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.5 Fixed typos and fixed CDN link issue.

v1.4 Updated the graphic to version v1.3.

Short (Article): https://fearby.com/go2/so/

Short (Image): https://fearby.com/go2/so-img/

Quick guide to using Adobe XD CC to design a prototype iOS app.

October 25, 2017 by Simon

Adobe has introduced (v1.0.x) Adobe XD CC, Adobe claims you can turn your best ideas into beautiful experiences — fast. Let’s give it a try.

Adobe Experience Design (Beta) is now Adobe XD CC. You can now design, prototype, and share amazing user experiences for websites, mobile apps, and more — all in the same app. Adobe XD CC is similar to Balsamic Mockup software.

Adobe XD Intro

Here is a great video demoing Adobe XD.

Install Adobe XD

If you don’t already have Adobe CC installed you can download a trial here. If you are wanting to install on Windows you will need Windows 10 (Anniversary Edition). Adobe has minimum system requirements listed here.

Start a Project

After you start an iOS project you will be looking for controls to add to your prototype. Adobe XD CC offers where you can download UI Kits direct from vendors (a shame when you are used to XCode or Visual Studio having controls preloaded).

Installing the Apple UI Design Resources

You will need to download the Apple UI Design Resources for Design XD from the Apple site (use the menu in the screenshot below or click here), they do not come with Adobe XD CC.

Here is more information on using Adobe XD CC UI Kits.

Download the iOS 11 UI resources for Adobe XD CC from the Apple site.

You can now extract the iOS resource files from Apple for use in Adobe XD projects. When iOS 12 and Android 9 comes out you can download new UI Kits.

Once you extract the files from the zip file, run the ./iOS-11-AdobeXD/Fonts/San Francisco Pro.pkg file to install iOS 11 font on yoir system.

I could not find a way to install the UI Kits permanently into Adobe XD CC (Searching revealed you need to open templates (as a separate process or open file in Adobe XD (double-click on the file)) and paste elements into your project). This seems clunky.

Why use Adobe XD

You can use Adobe XD to prototype interfaces around the common activities, a person may perform while using the apps you are prototyping. You can design an app’s onboarding, intro or user screens before actually developing the app.

http://bundle.greatsimple.io/

https://platforma.ws/ also has an extension for Adobe XD to allow you to get a prototype fast with ready to go layout elements. I will write a new blog post using https://platforma.ws/ in Adobe XD.

iOS Prototype Project

Let’s create an iOS project. Start a new iPhone 6/7 Project AND open up a UI template file in a second Adobe XD program (e.g ./iOS-11-AdobeXD/UI Elements + Design Templates + Guides/UIElements+DesignTemplates+Guides.xd).

Now you can drag and drop elements from the UI template (from Apple) into an XD CC app prototype project

TIP: Apple has a great site explaining how you can design and deliver apps (open the Apple Human Interfaces – iOS Design Themes page here). Apple also has assets and guidelines available for marketing your apps here.

To make buttons interactive you will need to click the Prototype tab and then drag the blue tabs to the right of interactive elements to the target screens.

You can learn more on making interactive prototypes here.

Tip: Don’t forget to add interactive links back to the home screen.

You can then press the play button to preview the app prototype simulated in software.

Export

You can now save and export your prototype app project to PNG, PDF, Web or other formats to others to send for review.

Adobe XD is big on saving to the Adobe Cloud allowing others to see changes in real-time. If you have linked assets in your prototype project (say Photoshop files) anyone viewing an XD prototype on the Adobe Cloud can automatically see changes in real-time (see then Adobe XD intro video above).

Running Prototypes on Real Devices

I was able to install Adobe XD app onto iOS, log in with my Adobe ID and the prototype popped up when I connected my iOS device to my Mac. More info here.

I was able to install the Android Adobe XD app and also sync a prototype app (Android was a bit slower to find the project but still the same process as iOS).

More Help

Adobe XD CC Official User Guide

https://helpx.adobe.com/xd/user-guide.html

30 Adobe XD CC/Adobe Comp tablet app tips

Conclusion

Pros

Adobe XD comes with Adobe CC.
Ope to feature enhancements.
Loads or 3rd party tools and user forums.
Automatic detection of duplicate actions (copy and paste grid items) and suggestion of repeating grids by pressing Command+R.

Cons

Unable to import UI Kits permanently into Adobe XD (I have to run multiple XD apps and paste UI elements between). Why would I no just stick with Adobe Photoshop?
Placement of UI elements like fonts feels clunky when compared to XCode and Visual Studio.
Duplicating prototype forms was not an option in the right-click (copy and Paste worked and so did ALT+Drag).

On the positive side, Adobe is openly allowing people to suggest and vote on features here https://adobexd.uservoice.com

But with Adobe XD you have the flexibility of having a design and prototyping product in one package with new monthly features.

More to come.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.2 added https://platforma.ws/ information.

etc

Short: https://fearby.com/go2/prototype/

How to use Sublime Text editor locally to edit code files on a remote server via SSH

September 16, 2017 by Simon

This guide will show you how to use Sublime Text editor locally to edit code files on a remote server via SSH.

This guide assumes oy already have a working SSH connection between your Mac and your remote server (with no firewall issues) and have configured SSH keys via modifying to authorized_keys file to enable SSH access.

Need a server?

I now use UpCLoud for cloud servers as they are super fast (read the blog post here). Get $25 free credit by signing up at UpCloud using this link.

UpCloud is way faster than Vulr.

Setting up slower region-specific servers can be found here. Set up a Server on Vultr here for as low as $2.5 a month or set up a Server on Digital Ocean (and get the first 2 months free ($5/m server)). I have a guide on setting up a Vultr server here or Digital Ocean server here. Don’t forget to add a free LetsEncrypt SSL Certificate and secure the server (read more here and here).

Buy a domain name from Namecheap here.

Setting up your local machine

Open Sublime Text 3 and press COMMAND+SHIFT+P to bring up the command bar and type Install and click Package Control: Install Package and click it.

Wait a few seconds for the packages list to show and type “rsub”

Ok let’s make an SSH alias to your server on your Mac by typing “sudo nano ~/.ssh/config”

Make these changes

File contents:

host mysrv
HostName www.myserver.com
User thesshuser
RemoteForward 52698 localhost:52698

Now we can connect to the server via SSH by typing “ssh mysrv”

After typing the server’s password you will be connected to the ssh server

ssh mysrv
[email protected]'s password: 
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-87-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

0 packages can be updated.
0 updates are security updates.


You have new mail.
Last login: Sat Sep 16 12:51:35 2017 from xx.xx.xx.xx
[email protected]:~#

Now on your local Mac load the following page in a web browser (and review the code): https://raw.github.com/aurora/rmate/master/rmate and copy the contents to the clipboard.

On the remote server (the SSH one) type:

sudo nano /usr/local/bin/rmate

Now paste the contents or this page into nano editor and save it and exit nano.

Now run this chmod command to make the rmate file executable.

sudo chmod a+x /usr/local/bin/rmate

Now on the server, we can open any text file with rmate and have it open locally in Sublime via SSH. Yes, Open a file on a server and have it automatically open in locally 🙂

If you have many files to open then create a bash file to open files with rmate

sudo nano openfilesonmac.sh

Contents:

#!/bin/bash

rmate index.html 
rmate index1.html 
rmate index2.html 
rmate index3.html 
rmate index4.html 
rmate index5.html 
rmate index6.html 
rmate index7.html 
rmate index8.html 
rmate index9.html 
rmate index10.html

File permissions:

chmod +x openfilesonmac.sh

Now we can open may remote files locally by running the bash script.

All saves in Sublime locally are sent to the server 🙂

e.g

rmate /www/index.html
rmate /node/api/app01/app.js
rmate /www/dashboard/index.php

Still here, read more articles here or use the form below to ask a question or recommend an article.

Port Forwarding with vSSH on OSX

If you use a third party ssh program like vSSH you will also need to setup port forwarding to avoid this error

rmate test.txt
/usr/local/bin/rmate: connect: Connection refused
/usr/local/bin/rmate: line 384: /dev/tcp/localhost/52698: Connection refused
Unable to connect to TextMate on localhost:52698

How.

Now you can open remote files locally with SSH or vSSH too.

Donate and make this blog better

Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.4 Added UpCloud Info.

v1.3 vSSH Port forwarding.

Run an Ubuntu VM system audit with Lynis

September 11, 2017 by Simon

Following on from my Securing Ubuntu in the cloud blog post I have installed Lynis open source security audit tool to check out to the security of my server in the cloud.

Lynis is an open source security auditing tool. Used by system administrators, security professionals, and auditors, to evaluate the security defences of their Linux and Unix-based systems. It runs on the host itself, so it performs more extensive security scans than vulnerability scanners. https://cisofy.com/lynis and https://github.com/CISOfy/lynis.

It is easy to setup a server in the cloud (create a server on Vultr or Digital Ocean here). Guides on setting up servers exist ( setup up a Vultr VM and configure it and digital ocean server) but how about securing it? You can install a LetsEncrypt SSL certificate in minutes or setup Content Security Policy and Public Key Pinning but don’t forget to get an external in-depth review of the security of your server(s).

Lynis Security Auditing Tool

Preparing install location (for Lynis)

cd /
mkdir utils
cd utils/

Install Lynis

sudo git clone https://www.github.com/CISOfy/lynis
Cloning into 'lynis'...
remote: Counting objects: 8357, done.
remote: Compressing objects: 100% (45/45), done.
remote: Total 8357 (delta 28), reused 42 (delta 17), pack-reused 8295
Receiving objects: 100% (8357/8357), 3.94 MiB | 967.00 KiB/s, done.
Resolving deltas: 100% (6121/6121), done.
Checking connectivity... done.

Running a Lynus system scan

./lynis audit system -Q

Lynis Results 1/3 Output (removed sensitive output)

[ Lynis 2.5.5 ]

################################################################################
  Lynis comes with ABSOLUTELY NO WARRANTY. This is free software, and you are
  welcome to redistribute it under the terms of the GNU General Public License.
  See the LICENSE file for details about using this software.

  2007-2017, CISOfy - https://cisofy.com/lynis/
  Enterprise support available (compliance, plugins, interface and tools)
################################################################################


[+] Initializing program
------------------------------------
- Detecting OS...  [ DONE ]
- Checking profiles... [ DONE ]

  ---------------------------------------------------
  Program version:           2.5.5
  Operating system:          Linux
  Operating system name:     Ubuntu Linux
  Operating system version:  16.04
  Kernel version:            4.4.0
  Hardware platform:         x86_64
  Hostname:                  yourservername
  ---------------------------------------------------
  Profiles:                  /linis/lynis/default.prf
  Log file:                  /var/log/lynis.log
  Report file:               /var/log/lynis-report.dat
  Report version:            1.0
  Plugin directory:          ./plugins
  ---------------------------------------------------
  Auditor:                   [Not Specified]
  Test category:             all
  Test group:                all
  ---------------------------------------------------
- Program update status...  [ NO UPDATE ]

[+] System Tools
------------------------------------
- Scanning available tools...
- Checking system binaries...

[+] Plugins (phase 1)
------------------------------------
: plugins have more extensive tests and may take several minutes to complete - Plugin pam
    [..]
- Plugin systemd
    [................]

[+] Boot and services
------------------------------------
- Service Manager [ systemd ]
- Checking UEFI boot [ DISABLED ]
- Checking presence GRUB [ OK ]
- Checking presence GRUB2 [ FOUND ]
- Checking for password protection [ OK ]
- Check running services (systemctl) [ DONE ]
: found 24 running services
- Check enabled services at boot (systemctl) [ DONE ]
: found 30 enabled services
- Check startup files (permissions) [ OK ]

[+] Kernel
------------------------------------
- Checking default run level [ RUNLEVEL 5 ]
- Checking CPU support (NX/PAE)
 support: PAE and/or NoeXecute supported [ FOUND ]
- Checking kernel version and release [ DONE ]
- Checking kernel type [ DONE ]
- Checking loaded kernel modules [ DONE ]
active modules
- Checking Linux kernel configuration file [ FOUND ]
- Checking default I/O kernel scheduler [ FOUND ]
- Checking for available kernel update [ OK ]
- Checking core dumps configuration [ DISABLED ]
- Checking setuid core dumps configuration [ PROTECTED ]
- Check if reboot is needed [ NO ]

[+] Memory and Processes
------------------------------------
- Checking /proc/meminfo [ FOUND ]
- Searching for dead/zombie processes [ OK ]
- Searching for IO waiting processes [ OK ]

[+] Users, Groups and Authentication
------------------------------------
- Administrator accounts [ OK ]
- Unique UIDs [ OK ]
- Consistency of group files (grpck) [ OK ]
- Unique group IDs [ OK ]
- Unique group names [ OK ]
- Password file consistency [ OK ]
- Query system users (non daemons) [ DONE ]
- NIS+ authentication support [ NOT ENABLED ]
- NIS authentication support [ NOT ENABLED ]
- sudoers file [ FOUND ]
- Check sudoers file permissions [ OK ]
- PAM password strength tools [ OK ]
- PAM configuration files (pam.conf) [ FOUND ]
- PAM configuration files (pam.d) [ FOUND ]
- PAM modules [ FOUND ]
- LDAP module in PAM [ NOT FOUND ]
- Accounts without expire date [ OK ]
- Accounts without password [ OK ]
- Checking user password aging (minimum) [ DISABLED ]
- User password aging (maximum) [ DISABLED ]
- Checking expired passwords [ OK ]
- Checking Linux single user mode authentication [ OK ]
- Determining default umask
- umask (/etc/profile) [ NOT FOUND ]
- umask (/etc/login.defs) [ SUGGESTION ]
- umask (/etc/init.d/rc) [ SUGGESTION ]
- LDAP authentication support [ NOT ENABLED ]
- Logging failed login attempts [ ENABLED ]

[+] Shells
------------------------------------
- Checking shells from /etc/shells
: found 6 shells (valid shells: 6).
- Session timeout settings/tools [ NONE ]
- Checking default umask values
- Checking default umask in /etc/bash.bashrc [ NONE ]
- Checking default umask in /etc/profile [ NONE ]

[+] File systems
------------------------------------
- Checking mount points
- Checking /home mount point [ SUGGESTION ]
- Checking /tmp mount point [ SUGGESTION ]
- Checking /var mount point [ SUGGESTION ]
- Query swap partitions (fstab) [ NONE ]
- Testing swap partitions [ OK ]
- Testing /proc mount (hidepid) [ SUGGESTION ]
- Checking for old files in /tmp [ OK ]
- Checking /tmp sticky bit [ OK ]
- ACL support root file system [ ENABLED ]
- Mount options of / [ NON DEFAULT ]
- Checking Locate database [ FOUND ]
- Disable kernel support of some filesystems
- Discovered kernel modules: cramfs freevxfs hfs hfsplus jffs2 udf 

[+] Storage
------------------------------------
- Checking usb-storage driver (modprobe config) [ NOT DISABLED ]
- Checking USB devices authorization [ ENABLED ]
- Checking firewire ohci driver (modprobe config) [ DISABLED ]

[+] NFS
------------------------------------
- Check running NFS daemon [ NOT FOUND ]

[+] Name services
------------------------------------
- Searching DNS domain name [ UNKNOWN ]
- Checking /etc/hosts
- Checking /etc/hosts (duplicates) [ OK ]
- Checking /etc/hosts (hostname) [ OK ]
- Checking /etc/hosts (localhost) [ SUGGESTION ]
- Checking /etc/hosts (localhost to IP) [ OK ]

[+] Ports and packages
------------------------------------
- Searching package managers
- Searching dpkg package manager [ FOUND ]
- Querying package manager
- Query unpurged packages [ NONE ]
- Checking security repository in sources.list file [ OK ]
- Checking APT package database [ OK ]
- Checking vulnerable packages [ OK ]
- Checking upgradeable packages [ SKIPPED ]
- Checking package audit tool [ INSTALLED ]

[+] Networking
------------------------------------
- Checking IPv6 configuration [ ENABLED ]
 method [ AUTO ]
 only [ NO ]
- Checking configured nameservers
- Testing nameservers
: 108.xx.xx.xx [ OK ]
: 2001:xxx:xxx:xxx::6 [ OK ]
- Minimal of 2 responsive nameservers [ OK ]
- Checking default gateway [ DONE ]
- Getting listening ports (TCP/UDP) [ DONE ]
* Found 18 ports
- Checking promiscuous interfaces [ OK ]
- Checking waiting connections [ OK ]
- Checking status DHCP client [ NOT ACTIVE ]
- Checking for ARP monitoring software [ NOT FOUND ]

[+] Printers and Spools
------------------------------------
- Checking cups daemon [ NOT FOUND ]
- Checking lp daemon [ NOT RUNNING ]

[+] Software: e-mail and messaging
------------------------------------
- Sendmail status [ RUNNING ]

[+] Software: firewalls
------------------------------------
- Checking iptables kernel module [ FOUND ]
- Checking iptables policies of chains [ FOUND ]
- Checking for empty ruleset [ OK ]
- Checking for unused rules [ FOUND ]
- Checking host based firewall [ ACTIVE ]

[+] Software: webserver
------------------------------------
- Checking Apache (binary /usr/sbin/apache2) [ FOUND ]
: No virtual hosts found
* Loadable modules [ FOUND (106) ]
- Found 106 loadable modules 
- anti-DoS/brute force [ OK ]
- web application firewall [ OK ]
- Checking nginx [ FOUND ]
- Searching nginx configuration file [ FOUND ]
- Found nginx includes [ 2 FOUND ]
- Parsing configuration options
- /etc/nginx/nginx.conf
- /etc/nginx/sites-enabled/default
- SSL configured [ YES ]
- Ciphers configured [ YES ]
- Prefer server ciphers [ YES ]
- Protocols configured [ YES ]
- Insecure protocols found [ NO ]
- Checking log file configuration
- Missing log files (access_log) [ NO ]
- Disabled access logging [ NO ]
- Missing log files (error_log) [ NO ]
- Debugging mode on error_log [ NO ]

[+] SSH Support
------------------------------------
- Checking running SSH daemon [ FOUND ]
- Searching SSH configuration [ FOUND ]
- SSH option: AllowTcpForwarding [ SUGGESTION ]
- SSH option: ClientAliveCountMax [ SUGGESTION ]
- SSH option: ClientAliveInterval [ OK ]
- SSH option: Compression [ SUGGESTION ]
- SSH option: FingerprintHash [ OK ]
- SSH option: GatewayPorts [ OK ]
- SSH option: IgnoreRhosts [ OK ]
- SSH option: LoginGraceTime [ OK ]
- SSH option: LogLevel [ SUGGESTION ]
- SSH option: MaxAuthTries [ SUGGESTION ]
- SSH option: MaxSessions [ SUGGESTION ]
- SSH option: PermitRootLogin [ SUGGESTION ]
- SSH option: PermitUserEnvironment [ OK ]
- SSH option: PermitTunnel [ OK ]
- SSH option: Port [ SUGGESTION ]
- SSH option: PrintLastLog [ OK ]
- SSH option: Protocol [ OK ]
- SSH option: StrictModes [ OK ]
- SSH option: TCPKeepAlive [ SUGGESTION ]
- SSH option: UseDNS [ OK ]
- SSH option: VerifyReverseMapping [ NOT FOUND ]
- SSH option: X11Forwarding [ SUGGESTION ]
- SSH option: AllowAgentForwarding [ SUGGESTION ]
- SSH option: AllowUsers [ NOT FOUND ]
- SSH option: AllowGroups [ NOT FOUND ]

[+] SNMP Support
------------------------------------
- Checking running SNMP daemon [ NOT FOUND ]

[+] Databases
------------------------------------
- MySQL process status [FOUND ]

[+] LDAP Services
------------------------------------
- Checking OpenLDAP instance [ NOT FOUND ]

[+] PHP
------------------------------------
- Checking PHP [ FOUND ]
- Checking PHP disabled functions [ FOUND ]
- Checking expose_php option [ OFF ]
- Checking enable_dl option [ OFF ]
- Checking allow_url_fopen option [ ON ]
- Checking allow_url_include option [ OFF ]
- Checking PHP suhosin extension status [ OK ]
- Suhosin simulation mode status [ OK ]

[+] Squid Support
------------------------------------
- Checking running Squid daemon [ NOT FOUND ]

[+] Logging and files
------------------------------------
- Checking for a running log daemon [ OK ]
- Checking Syslog-NG status [ NOT FOUND ]
- Checking systemd journal status [ FOUND ]
- Checking Metalog status [ NOT FOUND ]
- Checking RSyslog status [ FOUND ]
- Checking RFC 3195 daemon status [ NOT FOUND ]
- Checking minilogd instances [ NOT FOUND ]
- Checking logrotate presence [ OK ]
- Checking log directories (static list) [ DONE ]
- Checking open log files [ DONE ]
- Checking deleted files in use [ FILES FOUND ]

[+] Insecure services
------------------------------------
- Checking inetd status [ NOT ACTIVE ]

[+] Banners and identification
------------------------------------
- /etc/issue [ FOUND ]
- /etc/issue contents [ OK ]
- /etc/issue.net [ FOUND ]
- /etc/issue.net contents [ OK ]

[+] Scheduled tasks
------------------------------------
- Checking crontab/cronjob [ DONE ]
- Checking atd status [ RUNNING ]
- Checking at users [ DONE ]
- Checking at jobs [ NONE ]

[+] Accounting
------------------------------------
- Checking accounting information [ NOT FOUND ]
- Checking sysstat accounting data [ NOT FOUND ]
- Checking auditd [ NOT FOUND ]

[+] Time and Synchronization
------------------------------------
- NTP daemon found: ntpd [ FOUND ]
- NTP daemon found: systemd (timesyncd) [ FOUND ]
- Checking for a running NTP daemon or client [ OK ]
- Checking valid association ID's [ FOUND ]
- Checking high stratum ntp peers [ OK ]
- Checking unreliable ntp peers [ FOUND ]
- Checking selected time source [ OK ]
- Checking time source candidates [ OK ]
- Checking falsetickers [ OK ]
- Checking NTP version [ FOUND ]

[+] Cryptography
------------------------------------
- Checking for expired SSL certificates [0/1] [ NONE ]

[+] Virtualization
------------------------------------

[+] Containers
------------------------------------

[+] Security frameworks
------------------------------------
- Checking presence AppArmor [ FOUND ]
- Checking AppArmor status [ ENABLED ]
- Checking presence SELinux [ NOT FOUND ]
- Checking presence grsecurity [ NOT FOUND ]
- Checking for implemented MAC framework [ OK ]

[+] Software: file integrity
------------------------------------
- Checking file integrity tools
- Checking presence integrity tool [ NOT FOUND ]

[+] Software: System tooling
------------------------------------
- Checking automation tooling
- Automation tooling [ NOT FOUND ]
- Checking presence of Fail2ban [ FOUND ]
- Checking Fail2ban jails [ ENABLED ]
- Checking for IDS/IPS tooling [ FOUND ]

[+] Software: Malware
------------------------------------

[+] File Permissions
------------------------------------
- Starting file permissions check
/root/.ssh [ OK ]

[+] Home directories
------------------------------------
- Checking shell history files [ OK ]

[+] Kernel Hardening
------------------------------------
- Comparing sysctl key pairs with scan profile
- fs.protected_hardlinks (exp: 1) [ OK ]
- fs.protected_symlinks (exp: 1) [ OK ]
- fs.suid_dumpable (exp: 0) [ DIFFERENT ]
- kernel.core_uses_pid (exp: 1) [ DIFFERENT ]
- kernel.ctrl-alt-del (exp: 0) [ OK ]
- kernel.dmesg_restrict (exp: 1) [ DIFFERENT ]
- kernel.kptr_restrict (exp: 2) [ DIFFERENT ]
- kernel.randomize_va_space (exp: 2) [ OK ]
- kernel.sysrq (exp: 0) [ DIFFERENT ]
- net.ipv4.conf.all.accept_redirects (exp: 0) [ OK ]
- net.ipv4.conf.all.accept_source_route (exp: 0) [ OK ]
- net.ipv4.conf.all.bootp_relay (exp: 0) [ OK ]
- net.ipv4.conf.all.forwarding (exp: 0) [ OK ]
- net.ipv4.conf.all.log_martians (exp: 1) [ DIFFERENT ]
- net.ipv4.conf.all.mc_forwarding (exp: 0) [ OK ]
- net.ipv4.conf.all.proxy_arp (exp: 0) [ OK ]
- net.ipv4.conf.all.rp_filter (exp: 1) [ OK ]
- net.ipv4.conf.all.send_redirects (exp: 0) [ DIFFERENT ]
- net.ipv4.conf.default.accept_redirects (exp: 0) [ OK ]
- net.ipv4.conf.default.accept_source_route (exp: 0) [ OK ]
- net.ipv4.conf.default.log_martians (exp: 1) [ DIFFERENT ]
- net.ipv4.icmp_echo_ignore_broadcasts (exp: 1) [ OK ]
- net.ipv4.icmp_ignore_bogus_error_responses (exp: 1) [ OK ]
- net.ipv4.tcp_syncookies (exp: 1) [ DIFFERENT ]
- net.ipv4.tcp_timestamps (exp: 0) [ DIFFERENT ]
- net.ipv6.conf.all.accept_redirects (exp: 0) [ OK ]
- net.ipv6.conf.all.accept_source_route (exp: 0) [ OK ]
- net.ipv6.conf.default.accept_redirects (exp: 0) [ OK ]
- net.ipv6.conf.default.accept_source_route (exp: 0) [ OK ]

[+] Hardening
------------------------------------
- Installed compiler(s) [ FOUND ]
- Installed malware scanner [ NOT FOUND ]

[+] Custom Tests
------------------------------------
- Running custom tests...  [ NONE ]

[+] Plugins (phase 2)
------------------------------------
- Plugins (phase 2) [ DONE ]

================================================================================

...

Lynis Results 2/3 – Warnings

  Warnings (1):
  ----------------------------
  ! Found one or more vulnerable packages. [REMOVED-FIXED] 
      https://cisofy.com/controls/REMOVED-FIXED/
...

I resolved the only warning by typing

apt-get update
apt-get upgrade
shutdown -r now

After updating the Lynis system scan I re-ran the text and got

 -[ Lynis 2.5.5 Results ]-

  Great, no warnings

Lynis Results 3/3 – Suggestions

  Suggestions (44):
  ----------------------------
  * Set a password on GRUB bootloader to prevent altering boot configuration (e.g. boot in single user mode without password) [BOOT-5122] 
      https://cisofy.com/controls/BOOT-5122/

  * Configure minimum password age in /etc/login.defs [AUTH-9286] 
      https://cisofy.com/controls/AUTH-9286/

  * Configure maximum password age in /etc/login.defs [AUTH-9286] 
      https://cisofy.com/controls/AUTH-9286/

  * Default umask in /etc/login.defs could be more strict like 027 [AUTH-9328] 
      https://cisofy.com/controls/AUTH-9328/

  * Default umask in /etc/init.d/rc could be more strict like 027 [AUTH-9328] 
      https://cisofy.com/controls/AUTH-9328/

  * To decrease the impact of a full /home file system, place /home on a separated partition [FILE-6310] 
      https://cisofy.com/controls/FILE-6310/

  * To decrease the impact of a full /tmp file system, place /tmp on a separated partition [FILE-6310] 
      https://cisofy.com/controls/FILE-6310/

  * To decrease the impact of a full /var file system, place /var on a separated partition [FILE-6310] 
      https://cisofy.com/controls/FILE-6310/

  * Disable drivers like USB storage when not used, to prevent unauthorized storage or data theft [STRG-1840] 
      https://cisofy.com/controls/STRG-1840/

  * Check DNS configuration for the dns domain name [NAME-4028] 
      https://cisofy.com/controls/NAME-4028/

  * Split resolving between localhost and the hostname of the system [NAME-4406] 
      https://cisofy.com/controls/NAME-4406/

  * Install debsums utility for the verification of packages with known good database. [PKGS-7370] 
      https://cisofy.com/controls/PKGS-7370/

  * Update your system with apt-get update, apt-get upgrade, apt-get dist-upgrade and/or unattended-upgrades [PKGS-7392] 
      https://cisofy.com/controls/PKGS-7392/

  * Install package apt-show-versions for patch management purposes [PKGS-7394] 
      https://cisofy.com/controls/PKGS-7394/

  * Consider running ARP monitoring software (arpwatch,arpon) [NETW-3032] 
      https://cisofy.com/controls/NETW-3032/

  * Check iptables rules to see which rules are currently not used [FIRE-4513] 
      https://cisofy.com/controls/FIRE-4513/

  * Install Apache mod_evasive to guard webserver against DoS/brute force attempts [HTTP-6640] 
      https://cisofy.com/controls/HTTP-6640/

  * Install Apache modsecurity to guard webserver against web application attacks [HTTP-6643] 
      https://cisofy.com/controls/HTTP-6643/

  * Consider hardening SSH configuration [SSH-7408] 
    - Details  : AllowTcpForwarding (YES --> NO)
      https://cisofy.com/controls/SSH-7408/

  * Consider hardening SSH configuration [SSH-7408] 
    - Details  : ClientAliveCountMax (3 --> 2)
      https://cisofy.com/controls/SSH-7408/

  * Consider hardening SSH configuration [SSH-7408] 
    - Details  : Compression (DELAYED --> NO)
      https://cisofy.com/controls/SSH-7408/

  * Consider hardening SSH configuration [SSH-7408] 
    - Details  : LogLevel (INFO --> VERBOSE)
      https://cisofy.com/controls/SSH-7408/

  * Consider hardening SSH configuration [SSH-7408] 
    - Details  : MaxAuthTries (2 --> 1)
      https://cisofy.com/controls/SSH-7408/

  * Consider hardening SSH configuration [SSH-7408] 
    - Details  : MaxSessions (10 --> 2)
      https://cisofy.com/controls/SSH-7408/

  * Consider hardening SSH configuration [SSH-7408] 
    - Details  : PermitRootLogin (YES --> NO)
      https://cisofy.com/controls/SSH-7408/

  * Consider hardening SSH configuration [SSH-7408] 
    - Details  : Port (22 --> )
      https://cisofy.com/controls/SSH-7408/

  * Consider hardening SSH configuration [SSH-7408] 
    - Details  : TCPKeepAlive (YES --> NO)
      https://cisofy.com/controls/SSH-7408/

  * Consider hardening SSH configuration [SSH-7408] 
    - Details  : X11Forwarding (YES --> NO)
      https://cisofy.com/controls/SSH-7408/

  * Consider hardening SSH configuration [SSH-7408] 
    - Details  : AllowAgentForwarding (YES --> NO)
      https://cisofy.com/controls/SSH-7408/

  * Change the allow_url_fopen line to: allow_url_fopen = Off, to disable downloads via PHP [PHP-2376] 
      https://cisofy.com/controls/PHP-2376

  * Check what deleted files are still in use and why. [LOGG-2190] 
      https://cisofy.com/controls/LOGG-2190/

  * Add a legal banner to /etc/issue, to warn unauthorized users [BANN-7126] 
      https://cisofy.com/controls/BANN-7126/

  * Add legal banner to /etc/issue.net, to warn unauthorized users [BANN-7130] 
      https://cisofy.com/controls/BANN-7130/

  * Enable process accounting [ACCT-9622] 
      https://cisofy.com/controls/ACCT-9622/

  * Enable sysstat to collect accounting (no results) [ACCT-9626] 
      https://cisofy.com/controls/ACCT-9626/

  * Enable auditd to collect audit information [ACCT-9628] 
      https://cisofy.com/controls/ACCT-9628/

  * Check ntpq peers output for unreliable ntp peers and correct/replace them [TIME-3120] 
      https://cisofy.com/controls/TIME-3120/

  * Install a file integrity tool to monitor changes to critical and sensitive files [FINT-4350] 
      https://cisofy.com/controls/FINT-4350/

  * Determine if automation tools are present for system management [TOOL-5002] 
      https://cisofy.com/controls/TOOL-5002/

  * One or more sysctl values differ from the scan profile and could be tweaked [KRNL-6000] 
      https://cisofy.com/controls/KRNL-6000/

  * Harden compilers like restricting access to root user only [HRDN-7222] 
      https://cisofy.com/controls/HRDN-7222/

  * Harden the system by installing at least one malware scanner, to perform periodic file system scans [HRDN-7230] 
    - Solution : Install a tool like rkhunter, chkrootkit, OSSEC
      https://cisofy.com/controls/HRDN-7230/

  Follow-up
  ----------------------------
  - Show details of a test (lynis show details TEST-ID)
  - Check the logfile for all details (less /var/log/lynis.log)
  - Read security controls texts (https://cisofy.com)
  - Use --upload to upload data to central system (Lynis Enterprise users)

================================================================================

  Lynis security scan details

  Hardening index : 64 [############        ]
  Tests performed : 255
  Plugins enabled : 2

  Components
  - Firewall               [V]
  - Malware scanner        [X]

  Lynis Modules
  - Compliance Status      [?]
  - Security Audit         [V]
  - Vulnerability Scan     [V]

  Files
  - Test and debug information      : /var/log/lynis.log
  - Report data                     : /var/log/lynis-report.dat

================================================================================

  Lynis 2.5.5

  Auditing, system hardening, and compliance for UNIX-based systems
  (Linux, macOS, BSD, and others)

  2007-2017, CISOfy - https://cisofy.com/lynis/
  Enterprise support available (compliance, plugins, interface and tools)

================================================================================

  [TIP] Enhance Lynis audits by adding your settings to custom.prf (see /linis/lynis/default.prf for all settings)

Installing a Malware Scanner

Install ClamAV

sudo apt-get install clamav

Download virus and malware definitions (this takes about 30 min)

sudo freshclam

Output:

sudo freshclam
> ClamAV Update process started at Wed Nov 15th 20:44:55 2017
> Downloading main.cvd [10%]

I had an issue on some boxes with clamav reporting I could not run freshclam

sudo freshclam
ERROR: /var/log/clamav/freshclam.log is locked by another process
ERROR: Problem with internal logger (UpdateLogFile = /var/log/clamav/freshclam.log).

This was fixed by typing

rm -rf /var/log/clamav/freshclam.log
sudo freshclam

Troubleshooting clamav

Clam AV does not like low ram boxes and may produce this error

Downloading main.cvd [100%]
ERROR: Database load killed by signal 9
ERROR: Failed to load new database

It looks like the solution is to increase your total ram.

fyi: Scan with ClamAV

sudo clamscan --max-filesize=3999M --max-scansize=3999M --exclude-dir=/www/* -i -r /

Re-running Lynis gave me the following malware status

- Malware scanner        [V]

Lynis Security rating

Hardening index : 69 [##############      ]

Installed

sudo apt-get install apt-show-versions
sudo apt-get install arpwatch
sudo apt-get install arpon

After re-running the test I got this Lynis security rating score (an improvement of 1)

Hardening index : 70 [#############       ]

Installed and configured debsums and auditd

sudo apt-get install debsums
sudo apt-get install audit

Now I get the following Lynis security rating score.

Hardening index : 71 [##############      ]

Conclusion

Lynis is great at performing an audit and recommending areas of work to allow you to harden your system (brute force protection, firewall, etc)

Security Don’ts

Never think you are done securing a system.

Security Do’s

Update Software (and remove software you do not use.)
Check Lynis Suggestions and try and resolve.
Security is an ongoing process, Do install a firewall, do ban bad IP’s, Do whitelist good IP’s, Do review Logs,
Do limit port access, make backups and keep on securing.

I will keep on securing and try and get remove all issues.

Read my past post on Securing Ubuntu in the cloud.

Scheduling an auto system updates is not enough in Ubuntu (as it is not recommended as the administrator should make decisions, not a scheduled job).

apt-get update
apt-get upgrade

fyi: CISOFY/Lynis do have paid subscriptions to have external scans of your servers: https://cisofy.com/pricing. (why upgrade?)

I will look into this feature soon.

Updating Lynis

I checked the official documentation and ran an update check

./lynis --check-update
This option is deprecated
Use: lynis update info

./lynis update info

 == Lynis ==

  Version            : 2.5.5
  Status             : Outdated
  Installed version  : 255
  Latest version     : 257
  Release date       : 2017-09-07
  Update location    : https://cisofy.com/lynis/


2007-2017, CISOfy - https://cisofy.com/lynis/

Not sure how to update?

./lynis update
Error: Need a target for update

Examples:
lynis update check
lynis update info

./lynis update check
status=outdated

I opened an issue about updating v2.5.5 here. I asked Twiter for help.

Official Response: https://packages.cisofy.com/community/#debian-ubuntu

Waiting..

I ended up deleting Lynis 2.5.5

ls -al
rm -R *
rm -rf *
rm -rf .git
rm -rf .gitignore
rm -rf .travis.yml
cd ..
rm -R lynis/
ls -al

Updated

./lynis update check
status=up-to-date

And reinstalled to v2.5.8

sudo git clone https://www.github.com/CISOfy/lynis

Output:

sudo git clone https://www.github.com/CISOfy/lynis
Cloning into 'lynis'...
remote: Counting objects: 8538, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 8538 (delta 0), reused 0 (delta 0), pack-reused 8534
Receiving objects: 100% (8538/8538), 3.96 MiB | 2.01 MiB/s, done.
Resolving deltas: 100% (6265/6265), done.
Checking connectivity... done.

Installing and managing WordPress with WP-CLI from the command line on Ubuntu

September 10, 2017 by Simon

Here is a really simple way to manually manage WordPress installations on Ubuntu.

My previous WordPress (and related) guides

My Server Setup Guides

Go to https://make.wordpress.org/cli/handbook/installing/ and read the install instructions.

How to install wp-cli on Ubuntu

First read this post too install WordPress from the command line. You should install WordPress here.

Telnet to your server (SSH)

cd /www
mkdir wp-cli
cd wp-cli
pwd
/www/wp-cli
curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4191k  100 4191k    0     0  4535k      0 --:--:-- --:--:-- --:--:-- 4536k

Did the file download?

ls wp-cli.phar -al
-rw-r--r-- 1 root root 4292472 Sep 10 13:29 wp-cli.phar

Execute

/usr/bin/php ./wp-cli.phar --info

Make the file executable and move it to /usr/local/bin/wp

chmod +x wp-cli.phar
sudo mv wp-cli.phar /usr/local/bin/wp

Now you can run wp-cli by typing the following from the command line

wp --info

Read the official wp-cli quick start guide https://make.wordpress.org/cli/handbook/quick-start/

Ever since I moved my WordPress to a self-managed server I have not been able to update my plugins and I have not installed an FTP server (by choice for security reasons).

Let’s see if we can update plugin with wp-cli

cd /www
wp plugin update --all

Yes, it worked

Enabling Maintenance mode...
Downloading update from https://downloads.wordpress.org/plugin/add-to-any.1.7.17.zip...
Unpacking the update...
Installing the latest version...
Removing the old version of the plugin...
Plugin updated successfully.
Downloading update from https://downloads.wordpress.org/plugin/contact-form-7.4.9.zip...
Unpacking the update...
Installing the latest version...
Removing the old version of the plugin...
Plugin updated successfully.
Downloading update from https://downloads.wordpress.org/plugin/ewww-image-optimizer.3.6.1.zip...
Downloading update from https://downloads.wordpress.org/plugin/ewww-image-optimizer.3.6.1.zip...
Unpacking the update...
Installing the latest version...
Removing the old version of the plugin...
Plugin updated successfully.
Downloading update from https://downloads.wordpress.org/plugin/ewww-image-optimizer-cloud.3.6.1.zip...
Unpacking the update...
Installing the latest version...
Removing the old version of the plugin...
Plugin updated successfully.
Downloading update from https://downloads.wordpress.org/plugin/better-wp-security.6.6.0.zip...
Unpacking the update...
Installing the latest version...
Removing the old version of the plugin...
Plugin updated successfully.
Downloading update from https://www.wponlinesupport.com/edd-sl/package_download/removedapikey...
Warning: Download failed. "Unauthorized"
Downloading update from https://downloads.wordpress.org/plugin/wordpress-seo.5.4.zip...
Unpacking the update...
Installing the latest version...
Removing the old version of the plugin...
Plugin updated successfully.
Disabling Maintenance mode...
+----------------------------+-------------+-------------+---------+
| name                       | old_version | new_version | status  |
+----------------------------+-------------+-------------+---------+
| add-to-any                 | 1.7.16      | 1.7.17      | Updated |
| contact-form-7             | 4.8.1       | 4.9         | Updated |
| ewww-image-optimizer       | 3.5.1       | 3.6.1       | Updated |
| ewww-image-optimizer-cloud | 3.5.1       | 3.6.1       | Updated |
| better-wp-security         | 6.3.0       | 6.6.0       | Updated |
| wp-news-and-widget-pro     | 1.1.9       | 2.1.1       | Updated |
| wordpress-seo              | 5.1         | 5.4         | Updated |
+----------------------------+-------------+-------------+---------+
Success: Updated 7 of 7 plugins.

I have now updated all active plugins from one command line.

I have been unable to update WordPress itself since I moved my website (without an FTP server).

Let’s try and update WordPress with wp-cli.

Run this command.

cd /www
sudo wp core update

Success

Updating to version 4.8.1 (en_US)...
Downloading update from https://downloads.wordpress.org/release/wordpress-4.8.1-no-content.zip...
Unpacking the update...
Success: WordPress updated successfully.

Awesome 🙂

Now time to create a few executable bash files to update plugins and WordPress files in future.

Update WordPress Plugins Bash Script

sudo nano /scripts/updatewordpressplugins.sh

Contents

#!/bin/bash
cd /www
wp plugin update --all

Make Executable

chmod +X /scripts/updatewordpressplugins.sh

Update WordPress Bash Script

sudo nano /scripts/updatewordpress.sh

Contents

!/bin/bash
cd /www
sudo wp core update

Make Executable

chmod +X /scripts/updatewordpress.sh

More Commands

More commands can be found here: https://developer.wordpress.org/cli/commands/

Information on exporting tables can be found here https://developer.wordpress.org/cli/commands/db/export/

Site Wide Search and Replace

wp search-replace 'http://fearby.com' 'https://fearby.com' --dry-run

Results

+------------------+-----------------------+--------------+------+
| Table            | Column                | Replacements | Type |
+------------------+-----------------------+--------------+------+
| wp_commentmeta   | meta_key              | 0            | SQL  |
| wp_commentmeta   | meta_value            | 0            | SQL  |
| wp_comments      | comment_author        | 0            | SQL  |
| wp_comments      | comment_author_email  | 0            | SQL  |
| wp_comments      | comment_author_url    | 72           | SQL  |
| wp_comments      | comment_author_IP     | 0            | SQL  |
| wp_comments      | comment_content       | 0            | SQL  |
| wp_comments      | comment_approved      | 0            | SQL  |
| wp_comments      | comment_agent         | 0            | SQL  |
| wp_comments      | comment_type          | 0            | SQL  |
| wp_links         | link_url              | 0            | SQL  |
| wp_links         | link_name             | 0            | SQL  |
| wp_links         | link_image            | 0            | SQL  |
| wp_links         | link_target           | 0            | SQL  |
| wp_links         | link_description      | 0            | SQL  |
| wp_links         | link_visible          | 0            | SQL  |
| wp_links         | link_rel              | 0            | SQL  |
| wp_links         | link_notes            | 0            | SQL  |
| wp_links         | link_rss              | 0            | SQL  |
| wp_options       | option_name           | 0            | SQL  |
| wp_options       | option_value          | 10           | PHP  |
| wp_options       | autoload              | 0            | SQL  |
| wp_postmeta      | meta_key              | 0            | SQL  |
| wp_postmeta      | meta_value            | 56           | PHP  |
| wp_posts         | post_content          | 2176         | SQL  |
| wp_posts         | post_title            | 0            | SQL  |
| wp_posts         | post_excerpt          | 0            | SQL  |
| wp_posts         | post_status           | 0            | SQL  |
| wp_posts         | comment_status        | 0            | SQL  |
| wp_posts         | ping_status           | 0            | SQL  |
| wp_posts         | post_password         | 0            | SQL  |
| wp_posts         | post_name             | 0            | SQL  |
| wp_posts         | to_ping               | 0            | SQL  |
| wp_posts         | pinged                | 26           | SQL  |
| wp_posts         | post_content_filtered | 0            | SQL  |
| wp_posts         | guid                  | 3928         | SQL  |
| wp_posts         | post_type             | 0            | SQL  |
| wp_posts         | post_mime_type        | 0            | SQL  |
| wp_term_taxonomy | taxonomy              | 0            | SQL  |
| wp_term_taxonomy | description           | 0            | SQL  |
| wp_termmeta      | meta_key              | 0            | SQL  |
| wp_termmeta      | meta_value            | 0            | SQL  |
| wp_terms         | name                  | 0            | SQL  |
| wp_terms         | slug                  | 0            | SQL  |
| wp_usermeta      | meta_key              | 0            | SQL  |
| wp_usermeta      | meta_value            | 0            | PHP  |
| wp_users         | user_login            | 0            | SQL  |
| wp_users         | user_nicename         | 0            | SQL  |
| wp_users         | user_email            | 0            | SQL  |
| wp_users         | user_url              | 0            | SQL  |
| wp_users         | user_activation_key   | 0            | SQL  |
| wp_users         | display_name          | 0            | SQL  |
+------------------+-----------------------+--------------+------+
Success: 6268 replacements to be made.

If you are updating past pages and post links do include https look for www alos

wp search-replace 'http://www.fearby.com' 'https://www.fearby.com' --dry-run

+------------------+-----------------------+--------------+------+
| Table            | Column                | Replacements | Type |
+------------------+-----------------------+--------------+------+
| wp_commentmeta   | meta_key              | 0            | SQL  |
| wp_commentmeta   | meta_value            | 0            | SQL  |
| wp_comments      | comment_author        | 0            | SQL  |
| wp_comments      | comment_author_email  | 0            | SQL  |
| wp_comments      | comment_author_url    | 0            | SQL  |
| wp_comments      | comment_author_IP     | 0            | SQL  |
| wp_comments      | comment_content       | 0            | SQL  |
| wp_comments      | comment_approved      | 0            | SQL  |
| wp_comments      | comment_agent         | 0            | SQL  |
| wp_comments      | comment_type          | 0            | SQL  |
| wp_links         | link_url              | 0            | SQL  |
| wp_links         | link_name             | 0            | SQL  |
| wp_links         | link_image            | 0            | SQL  |
| wp_links         | link_target           | 0            | SQL  |
| wp_links         | link_description      | 0            | SQL  |
| wp_links         | link_visible          | 0            | SQL  |
| wp_links         | link_rel              | 0            | SQL  |
| wp_links         | link_notes            | 0            | SQL  |
| wp_links         | link_rss              | 0            | SQL  |
| wp_options       | option_name           | 0            | SQL  |
| wp_options       | option_value          | 1            | PHP  |
| wp_options       | autoload              | 0            | SQL  |
| wp_postmeta      | meta_key              | 0            | SQL  |
| wp_postmeta      | meta_value            | 42           | PHP  |
| wp_posts         | post_content          | 729          | SQL  |
| wp_posts         | post_title            | 3            | SQL  |
| wp_posts         | post_excerpt          | 0            | SQL  |
| wp_posts         | post_status           | 0            | SQL  |
| wp_posts         | comment_status        | 0            | SQL  |
| wp_posts         | ping_status           | 0            | SQL  |
| wp_posts         | post_password         | 0            | SQL  |
| wp_posts         | post_name             | 0            | SQL  |
| wp_posts         | to_ping               | 0            | SQL  |
| wp_posts         | pinged                | 10           | SQL  |
| wp_posts         | post_content_filtered | 0            | SQL  |
| wp_posts         | guid                  | 0            | SQL  |
| wp_posts         | post_type             | 0            | SQL  |
| wp_posts         | post_mime_type        | 0            | SQL  |
| wp_term_taxonomy | taxonomy              | 0            | SQL  |
| wp_term_taxonomy | description           | 0            | SQL  |
| wp_termmeta      | meta_key              | 0            | SQL  |
| wp_termmeta      | meta_value            | 0            | SQL  |
| wp_terms         | name                  | 0            | SQL  |
| wp_terms         | slug                  | 0            | SQL  |
| wp_usermeta      | meta_key              | 0            | SQL  |
| wp_usermeta      | meta_value            | 0            | PHP  |
| wp_users         | user_login            | 0            | SQL  |
| wp_users         | user_nicename         | 0            | SQL  |
| wp_users         | user_email            | 0            | SQL  |
| wp_users         | user_url              | 0            | SQL  |
| wp_users         | user_activation_key   | 0            | SQL  |
| wp_users         | display_name          | 0            | SQL  |
+------------------+-----------------------+--------------+------+

When you are ready to run the replace remove the

--dry-run

As always, backup your database and files before you make changes.

Success: Made 6270 replacements.
Success: Made 787 replacements.

This was the easiest way to update my WordPress site and force existing links use https links. Of course, you can downgrade https links if you don’t want to have https anymore.

Troubleshooting

wp needs to be run from the root folder of your web server (or it will report an error). You can specify a path location as a parameter too ( –path=’/path/to/wordpress’)

Error: This does not seem to be a WordPress install.

All the following the wp command if you are logged in as root (not recommended for security reasons).

--allow-root

Adding “Advertisement” text above Google ads in all WordPress pages and posts, read this guide on setting up AdSense on your blog (unless you go Auto Ads).

sudo wp search-replace '<script async src="//pagead2.googlesyndication.com' 'Advertisement:<br /> <script async="" src="//pagead2.googlesyndication.com'
sudo wp search-replace '<script async="" src="//pagead2.googlesyndication.com' 'Advertisement:<br /> <script async="" src="//pagead2.googlesyndication.com'

Official Troubleshooting guide here

I had an issue where I received error messages running the wp tool, iI fixed this editing my php.ini file and changing this..

memory_limit = 128

memory_limit = 128M

The M made all the difference after I restarted PHP/NGINX.

to this

Thank You, Kerry Hoath for the tip on this awesome cli tool.

fyi: WordPress fixed a SQL injection vulnerability and the wp-cli tool helped me update WordPress from the CLI.

Update Themes

sudo wp theme update --all --allow-root

Update Plugins

sudo wp plugins update --all --allow-root

Update WordPress Core

sudo wp core update --allow-root

Donate and make this blog better

Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.5 Updating WordPress themes, plugins or core

v1.42 added ‘cd /www’ before calling wp cli.

How to backup an Ubuntu VM in the cloud via crontab entries that trigger Bash Scripts, SSH, rsync and email backup alerts

August 20, 2017 by Simon

Here is how I backup a number of Ubuntu servers with crontab entries, bash scripts and rsync and send backup email.

Read more on useful terminal commands here for as low as $2.5 a month. Read on setting up a Digital Ocean Ubuntu server here for as low as $5 a month here ($10 free credit). Read more on setting up an AWS Ubuntu server here.

I have 6 numbered scripts in my scripts folder that handle backups, I call these scripts at set times via the crontab list.

fyi: Paths below have been changed for the purpose of this post (security).

1 1 * * * /bin/bash /scripts-folder/0.backupfiles.sh >> /backup-folder/0.backupfiles.log
3 1 * * * /bin/bash /scripts-folder/1.backupdbs.sh >> /backup-folder/1.backupdbs.log
5 1 * * * /bin/bash /scripts-folder/2.shrinkmysql.sh >> /backup-folder/2.shrinkmysql.log
10 1 * * * /bin/bash /scripts-folder/3.addtobackuplog.sh >> /backup-folder/3.addtobackuplog.log
11 1 * * * /bin/bash /scripts-folder/4.syncfiles.sh >> /backup-folder/4.syncfiles.log
15 1 * * * /bin/bash /scripts-folder/5.sendbackupemail.sh > /dev/null 2>&1

https://crontab.guru/ is great for specifying times to run jobs on each server (I backup one server at 1 AM,, another at 2 AM etc (never at the same time))

Bring up your crontab list

crontab -e

Check out the Crontab schedule generator here.

Below is the contents of my /scripts/0.backupfiles.sh (sensitive information removed).

I use this script to backup folders and configuration data

cat /scripts-folder/0.backupfiles.sh
#!/bin/bash

echo "Deleting old NGINX config..";
rm /backup-folder/config-nginx.zip

echo "Backing Up NGNIX..";
zip -r -9 /backup-folder/config-nginx.zip /etc/nginx/ -x "*.tmp" -x "*.temp" -x"./backup-folder/*.bak" -x "./backup-folder/*.zip"

echo "Deleting old www backup(s) ..";
#rm /backup-folder/www.zip
echo "Removing old www backup folder";
rm -R /backup-folder/www
echo "Making new backup folder at /backup-folder/www/";
mkdir /backup-folder/www

echo "Copying /www/ to /backup-folder/www/";
cp -rTv /www/ /backup-folder/www/
echo "Done copying /www/ to /backup-folder/www/";

Below is the contents of my /scripts-folder/1.backupdbs.sh (sensitive information removed).

I use this script to dump my MySQL database.

cat /scripts-folder/1.backupdbs.sh
#!/bin/bash

echo "$(date) 1.backupdbs.sh ...." >> /backup-folder/backup.log

echo "Removing old SQL backup..":
rm /backup-folder/mysql/database-dump.sql

echo "Backing up SQL";
/usr/bin/mysqldump --all-databases > /backup-folder/mysql/database-dump.sql -u 'mysqluser' -p'[email protected]$word'

echo "Done backing up the database";

Below is the contents of my /scripts-folder/2.shrinkmysql.sh (sensitive information removed).

I use this script to tar my SQL dumps as these files can be quite big

cat /scripts-folder/2.shrinkmysql.sh
#!/bin/bash

echo "$(date) 2.shrinkmysql.sh ...." >> /backup-folder/backup.log

echo "Backing up MySQL dump..";
tar -zcf /backup-folder/mysql.tgz /backup-folder/mysql/

echo "Removing old MySQL dump..";
rm /backup-folder/mysql/*.sql

Below is the contents of my /scripts-folder/3.addtobackuplog.sh (sensitive information removed).

This script is handy for dumping extra information.

cat /scripts-folder/3.addtobackuplog.sh
#!/bin/bash

echo "$(date) 3.addtobackuplog.sh ...." >> /backup-folder/backup.log

echo "Server Name.." >> /backup-folder/backup.log
grep "server_name" /etc/nginx/sites-available/default

echo "$(date) Timec" >> /backup-folder/backup.log
sudo hwclock --show  >> /backup-folder/backup.log

echo "$(date) Uptime, Load etc" >> /backup-folder/backup.log
w -i >> /backup-folder/backup.log

echo "$(date) Memory" >> /backup-folder/backup.log
free  >> /backup-folder/backup.log

echo "$(date) Disk Space" >> /backup-folder/backup.log
pydf >> /backup-folder/backup.log

echo "Firewall" >> /backup-folder/backup.log
ufw status >> /backup-folder/backup.log

echo "Adding to Backup Log file..";
echo "$(date) Nightly MySQL Backup Successful....." >> /backup-folder/backup.log

Below is the contents of my /scripts-folder/4.syncfiles.sh (sensitive information removed).

This script is the workhorse routine that rsyncs files to the source to the backup server (a dedicated Vulr server with an A Name record attaching the server to my domain).

I installed sshpass to pass in the ssh user password (after ssh is connected (authorized_keys set), I tried to setup a rsync daemon but had no luck). I ensured appropriate ports were opened on the source (OUT 22, 873) and backup server (IN 22 873).

cat /scripts-folder/4.syncfiles.sh
#!/bin/bash

echo "$(date) 4.syncfiles.sh ...." >> /backup-folder/backup.log
echo "Syncing Files.";

sudo sshpass -p 'Y0urW0rkingSSHR00tPa$0ord' rsync -a -e  'ssh -p 22 ' --progress -P /backup-folder backup-server.yourdomain.com:/backup-folder/1.www.server01.com/

ufw firewall has great rules for allowing certain IP’s to talk on ports.

Set Outbound firewall rules (to certain IP’s)

sudo ufw allow from 123.123.123.123 to any port 22

Change 123.123.123.123 to your backup server.

Set Inbound firewall rules (to certain IP’s)

sudo ufw allow out from 123.123.123.123 to any port 22

Change 123.123.123.123 to your sending server.

You can and should setup rate limits on IP’s hitting certain ports.

udo ufw limit 22 comment 'Rate limit for this port has been reached'

Install Fail2Ban to automatically ban certain users. Fail2Ban reads log file that contains password failure report
and bans the corresponding IP addresses using firewall rules. Read more on securing Ubuntu in the cloud here.

Below is the contents of my /scripts-folder/5.sendbackupemail.sh (sensitive information removed).

This script sends an email and attaches a zip file of all log files generated through the backup process.

cat /scripts/5.sendbackupemail.sh
#!/bin/bash

echo "$(date) 5.sendbackupemail.sh ...." >> /backup-folder/backup.log

echo "Zipping up log Files.";

zip -r -9 /backup-folder/backup-log.zip /backup-folder/*.log

echo "Sending Email";
sendemail -f [email protected] -t [email protected] -u "Backup Alert" -m "server01 has been backed up" -s smtp.gmail.com:587 -o tls=yes -xu [email protected] -xp Y0urGSu1tePasswordG0e$Here123 -a /backup-folder/backup-log.zip

Read my guide on setting up sendmail here.

Security Considerations

You should never store passwords in scripts that talk to SSH connections, create MySQL dumps or when talking to email servers, I will update this guide when I solving all of these cases. Also, create the least access required for user accounts where possible.

Target Server Configuration

Alos you can see in /scripts-folder/4.syncfiles.sh that I am saving to the ‘/backup-folder/1.www.server01.com/’ folder, you can make as many folders as you want to make the most of the backup server. I would advise you not use the server for anything else like web servers and apps as this server is holding important stuff.

backup-server.yourdomain.com:/backup-folder/1.www.server01.com/

I have a handy script to delete all backups (handy during testing).

#!/bin/bash

echo "Deleting Backup Folders..........................................";

echo " Deleting /backup-folder/1.www.server01.com";
rm -R /backup-folder/1.www.server01.com

echo " Deleting /backup-folder/2.www.server02.com";
rm -R /backup-folder/2.www.server02.com

echo " Deleting /backup-folder/3.www.server03.com";
rm -R /backup-folder/3.www.server03.com

echo " Deleting /backup-folder/4.www.server04.com";
rm -R /backup-folder/4.www.server04.com

echo " Deleting /backup-folder/5.www.server05.com";
rm -R /backup-folder/5.www.server05.com

echo " Deleting /backup-folder/6.www.server06.com";
rm -R /backup-folder/6.www.server06.com

echo " Deleting /backup-folder/7.www.server07.com";
rm -R /backup-folder/7.www.server07.com

echo " Deleting /backup-folder/8.www.server08.com";
rm -R /backup-folder/8.www.server08.com

echo "
";

echo "Creating Backup Folders.........................................";

echo " Making folder /backup-folder/1.www.server01.com";
mkdir /backup-folder/1.www.server01.com

echo " Making folder /backup-folder/2.www.server02.com";
mkdir /backup-folder/2.www.server02.com

echo " Making folder /backup-folder/3.www.server03.com";
mkdir /backup-folder/3.www.server03.com";

echo " Making folder /backup-folder/4.www.server04.com";
mkdir /backup-folder/4.www.server04.com

echo " Making folder /backup-folder/5.www.server04.com";
mkdir /backup-folder/5.www.server04.com

echo " Making folder /backup-folder/6.www.server05.com";
mkdir /backup-folder/6.www.server04.com

echo " Making folder /backup-folder/7.www.server06.com";
mkdir /backup-folder/7.www.server04.com

echo " Making folder /backup-folder/8.www.server07.com";
mkdir /backup-folder/8.www.server08.com

echo "
";

echo "Backup Folder Contents.........................................";
ls /backup-folder -al
echo "
";

echo "Folder Strcuture...............................................";
cd /backup-folder
pwd
tree -a -f -p -h  -l -R

echo "
";

echo "How big is the backup folder...................................";
du -hs /backup-folder

echo "
";

echo "Done...........................................................";

Ensure your backup server is just for backups and only allows traffic from known IP’s

ufw status
Status: active

To                         Action      From
--                         ------      ----
22                         ALLOW       123.123.123.123
22                         ALLOW       123.123.123.124
22                         ALLOW       123.123.123.125
22                         ALLOW       123.123.123.126
22                         ALLOW       123.123.123.127
22                         ALLOW       123.123.123.128
22                         ALLOW       123.123.123.129
22                         ALLOW       123.123.123.130
53                         ALLOW       Anywhere

22                         ALLOW OUT   123.123.123.123
22                         ALLOW OUT   123.123.123.124
22                         ALLOW OUT   123.123.123.125
22                         ALLOW OUT   123.123.123.126
22                         ALLOW OUT   123.123.123.127
22                         ALLOW OUT   123.123.123.128
22                         ALLOW OUT   123.123.123.129
22                         ALLOW OUT   123.123.123.130

Change the 123.x.x.x servers to your servers IP’s

Tip: Keep an eye on the backups with tools like ncdu

sudo ncdu /backup-folder
ncdu 1.11 ~ Use the arrow keys to navigate, press ? for help
--- /backup ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    1.0 GiB [##########] /6.www.server01.com
  462.1 MiB [####      ] /1.www.server02.com
  450.1 MiB [####      ] /5.www.server03.com
   60.1 MiB [          ] /2.www.server04.com
  276.0 KiB [          ] /3.www.server05.com
  276.0 KiB [          ] /4.www.server06.com
e   4.0 KiB [          ] /8.www.server07.com
e   4.0 KiB [          ] /7.www.server08.com

Installing SSH on OSX

If you want to backup to this server with OSyouou will need to install sshpass

curl -O -L http://downloads.sourceforge.net/project/sshpass/sshpass/1.06/sshpass-1.06.tar.gz && tar xvzf sshpass-1.06.tar.gz
cd sshpass-1.06
./configure
sudo make install

sshpass should be installed

sshpass -V
sshpass 1.06
(C) 2006-2011 Lingnu Open Source Consulting Ltd.
(C) 2015-2016 Shachar Shemesh
This program is free software, and can be distributed under the terms of the GPL
See the COPYING file for more information.

Using "assword" as the default password prompt indicator.

I have not got sshp[ass working yet error “Host key verification failed.” I had to remove the back known host from ~/.ssh/known_hosts” on OSX

But this worked on OSX

rsync -a -e 'ssh -p 22 ' --progress -P ~/Desktop [email protected]:/backup/8.Mac/

Note: Enter the servers [email protected] before the hostname or rsync will use the logged in OSX username

Don’t forget to check the backup serves disk usage often.

Output from backing up an incremental update (1x new folder)

localhost:~ local-account$ rsync -a -e  'ssh -p 22 ' --progress -P /Users/local-account/folder-to-backup [email protected]:/backup/the-computer/
[email protected]'s password: 
building file list ... 
51354 files to consider
folder-to-backup/
folder-to-backup/TestProject/
folder-to-backup/TestProject/.git/
folder-to-backup/TestProject/.git/COMMIT_EDITMSG
          15 100%    0.00kB/s    0:00:00 (xfer#1, to-check=16600/51354)
folder-to-backup/TestProject/.git/HEAD
          23 100%   22.46kB/s    0:00:00 (xfer#2, to-check=16599/51354)
folder-to-backup/TestProject/.git/config
         137 100%  133.79kB/s    0:00:00 (xfer#3, to-check=16598/51354)
folder-to-backup/TestProject/.git/description
          73 100%   10.18kB/s    0:00:00 (xfer#4, to-check=16597/51354)
folder-to-backup/TestProject/.git/index
        1581 100%  220.56kB/s    0:00:00 (xfer#5, to-check=16596/51354)
folder-to-backup/TestProject/.git/hooks/
folder-to-backup/TestProject/.git/hooks/README.sample
         177 100%   21.61kB/s    0:00:00 (xfer#6, to-check=16594/51354)
folder-to-backup/TestProject/.git/info/
folder-to-backup/TestProject/.git/info/exclude
          40 100%    4.88kB/s    0:00:00 (xfer#7, to-check=16592/51354)
folder-to-backup/TestProject/.git/logs/
folder-to-backup/TestProject/.git/logs/HEAD
         164 100%   20.02kB/s    0:00:00 (xfer#8, to-check=16590/51354)
folder-to-backup/TestProject/.git/logs/refs/
folder-to-backup/TestProject/.git/logs/refs/heads/
folder-to-backup/TestProject/.git/logs/refs/heads/master
         164 100%   20.02kB/s    0:00:00 (xfer#9, to-check=16587/51354)
folder-to-backup/TestProject/.git/objects/
folder-to-backup/TestProject/.git/objects/05/
folder-to-backup/TestProject/.git/objects/05/0853a802dd40cad0e15afa19516e9ad94f5801
        2714 100%  294.49kB/s    0:00:00 (xfer#10, to-check=16584/51354)
folder-to-backup/TestProject/.git/objects/11/
folder-to-backup/TestProject/.git/objects/11/729e81fc116908809fc17d60c8604aa43ec095
         105 100%   11.39kB/s    0:00:00 (xfer#11, to-check=16582/51354)
folder-to-backup/TestProject/.git/objects/23/
folder-to-backup/TestProject/.git/objects/23/768a20baaf8aa0c31b0e485612a5e245bb570d
         131 100%   12.79kB/s    0:00:00 (xfer#12, to-check=16580/51354)
folder-to-backup/TestProject/.git/objects/27/
folder-to-backup/TestProject/.git/objects/27/3375fc70381bd2608e05c03e00ee09c42bdc58
         783 100%   76.46kB/s    0:00:00 (xfer#13, to-check=16578/51354)
folder-to-backup/TestProject/.git/objects/2a/
folder-to-backup/TestProject/.git/objects/2a/507ef5ea3b1d68c2d92bb4aece950ef601543e
         303 100%   26.90kB/s    0:00:00 (xfer#14, to-check=16576/51354)
folder-to-backup/TestProject/.git/objects/2b/
folder-to-backup/TestProject/.git/objects/2b/f8bd93d56787a7548c7f8960a94f05c269b486
         136 100%   12.07kB/s    0:00:00 (xfer#15, to-check=16574/51354)
folder-to-backup/TestProject/.git/objects/2f/
folder-to-backup/TestProject/.git/objects/2f/900764e9d12d8da7e5e01ba34d2b7b2d95ffd4
         209 100%   17.01kB/s    0:00:00 (xfer#16, to-check=16572/51354)
folder-to-backup/TestProject/.git/objects/36/
folder-to-backup/TestProject/.git/objects/36/d2c80d8893178d7e1f2964085b273959bfdc28
         201 100%   16.36kB/s    0:00:00 (xfer#17, to-check=16570/51354)
folder-to-backup/TestProject/.git/objects/3d/
folder-to-backup/TestProject/.git/objects/3d/e5a02083dbe9c23731a38901dca9e913c04dd0
         130 100%   10.58kB/s    0:00:00 (xfer#18, to-check=16568/51354)
folder-to-backup/TestProject/.git/objects/40/
folder-to-backup/TestProject/.git/objects/40/40592d8d4d886a5c81e1369ddcde71dd3b66b5
         841 100%   63.18kB/s    0:00:00 (xfer#19, to-check=16566/51354)
folder-to-backup/TestProject/.git/objects/87/
folder-to-backup/TestProject/.git/objects/87/60f48ddbc9ed0863e3fdcfce5e4536d08f9b8d
          86 100%    6.46kB/s    0:00:00 (xfer#20, to-check=16564/51354)
folder-to-backup/TestProject/.git/objects/a9/
folder-to-backup/TestProject/.git/objects/a9/e6a23fa34a5de4cd36250dc0d797439d85f2ea
         306 100%   22.99kB/s    0:00:00 (xfer#21, to-check=16562/51354)
folder-to-backup/TestProject/.git/objects/b0/
folder-to-backup/TestProject/.git/objects/b0/4364089fdc64fe3b81bcd41462dd55edb7a001
          57 100%    4.28kB/s    0:00:00 (xfer#22, to-check=16560/51354)
folder-to-backup/TestProject/.git/objects/be/
folder-to-backup/TestProject/.git/objects/be/3b93d6d8896d69670f1a8e26d1f51f9743d07e
          60 100%    4.19kB/s    0:00:00 (xfer#23, to-check=16558/51354)
folder-to-backup/TestProject/.git/objects/d0/
folder-to-backup/TestProject/.git/objects/d0/524738680109d9f0ca001dad7c9bbf563e898e
         523 100%   36.48kB/s    0:00:00 (xfer#24, to-check=16556/51354)
folder-to-backup/TestProject/.git/objects/d5/
folder-to-backup/TestProject/.git/objects/d5/4e024fe16b73e5602934ef83e0b32a16243a5e
          69 100%    4.49kB/s    0:00:00 (xfer#25, to-check=16554/51354)
folder-to-backup/TestProject/.git/objects/db/
folder-to-backup/TestProject/.git/objects/db/3f0ce163c8033a175d27de6a4e96aadc115625
          59 100%    3.84kB/s    0:00:00 (xfer#26, to-check=16552/51354)
folder-to-backup/TestProject/.git/objects/df/
folder-to-backup/TestProject/.git/objects/df/cad4828b338206f0a7f18732c086c4ef959a7b
          51 100%    3.32kB/s    0:00:00 (xfer#27, to-check=16550/51354)
folder-to-backup/TestProject/.git/objects/ef/
folder-to-backup/TestProject/.git/objects/ef/e6d036f817624654f77c4a91ae6f20b5ecbe9d
          94 100%    5.74kB/s    0:00:00 (xfer#28, to-check=16548/51354)
folder-to-backup/TestProject/.git/objects/f2/
folder-to-backup/TestProject/.git/objects/f2/b43571ec42bad7ac43f19cf851045b04b6eb29
         936 100%   57.13kB/s    0:00:00 (xfer#29, to-check=16546/51354)
folder-to-backup/TestProject/.git/objects/fd/
folder-to-backup/TestProject/.git/objects/fd/f3f97d1b6e9d8d29bb69a88c4d89ca752bd937
         807 100%   49.26kB/s    0:00:00 (xfer#30, to-check=16544/51354)
folder-to-backup/TestProject/.git/objects/info/
folder-to-backup/TestProject/.git/objects/pack/
folder-to-backup/TestProject/.git/refs/
folder-to-backup/TestProject/.git/refs/heads/
folder-to-backup/TestProject/.git/refs/heads/master
          41 100%    2.50kB/s    0:00:00 (xfer#31, to-check=16539/51354)
folder-to-backup/TestProject/.git/refs/tags/
folder-to-backup/TestProject/TestProject.xcodeproj/
folder-to-backup/TestProject/TestProject.xcodeproj/project.pbxproj
       11476 100%  659.24kB/s    0:00:00 (xfer#32, to-check=16536/51354)
folder-to-backup/TestProject/TestProject.xcodeproj/project.xcworkspace/
folder-to-backup/TestProject/TestProject.xcodeproj/project.xcworkspace/contents.xcworkspacedata
         156 100%    8.96kB/s    0:00:00 (xfer#33, to-check=16534/51354)
folder-to-backup/TestProject/TestProject.xcodeproj/project.xcworkspace/xcuserdata/
folder-to-backup/TestProject/TestProject.xcodeproj/project.xcworkspace/xcuserdata/simon.xcuserdatad/
folder-to-backup/TestProject/TestProject.xcodeproj/project.xcworkspace/xcuserdata/simon.xcuserdatad/UserInterfaceState.xcuserstate
        8190 100%  470.47kB/s    0:00:00 (xfer#34, to-check=16531/51354)
folder-to-backup/TestProject/TestProject.xcodeproj/xcuserdata/
folder-to-backup/TestProject/TestProject.xcodeproj/xcuserdata/simon.xcuserdatad/
folder-to-backup/TestProject/TestProject.xcodeproj/xcuserdata/simon.xcuserdatad/xcschemes/
folder-to-backup/TestProject/TestProject.xcodeproj/xcuserdata/simon.xcuserdatad/xcschemes/TestProject.xcscheme
        3351 100%  192.50kB/s    0:00:00 (xfer#35, to-check=16527/51354)
folder-to-backup/TestProject/TestProject.xcodeproj/xcuserdata/simon.xcuserdatad/xcschemes/xcschememanagement.plist
         483 100%   27.75kB/s    0:00:00 (xfer#36, to-check=16526/51354)
folder-to-backup/TestProject/TestProject/
folder-to-backup/TestProject/TestProject/AppDelegate.swift
        2172 100%  117.84kB/s    0:00:00 (xfer#37, to-check=16524/51354)
folder-to-backup/TestProject/TestProject/Info.plist
        1442 100%   78.23kB/s    0:00:00 (xfer#38, to-check=16523/51354)
folder-to-backup/TestProject/TestProject/ViewController.swift
         505 100%   27.40kB/s    0:00:00 (xfer#39, to-check=16522/51354)
folder-to-backup/TestProject/TestProject/Assets.xcassets/
folder-to-backup/TestProject/TestProject/Assets.xcassets/AppIcon.appiconset/
folder-to-backup/TestProject/TestProject/Assets.xcassets/AppIcon.appiconset/Contents.json
        1077 100%   58.43kB/s    0:00:00 (xfer#40, to-check=16519/51354)
folder-to-backup/TestProject/TestProject/Base.lproj/
folder-to-backup/TestProject/TestProject/Base.lproj/LaunchScreen.storyboard
        1740 100%   94.40kB/s    0:00:00 (xfer#41, to-check=16517/51354)
folder-to-backup/TestProject/TestProject/Base.lproj/Main.storyboard
        1695 100%   91.96kB/s    0:00:00 (xfer#42, to-check=16516/51354)

sent 1243970 bytes  received 1220 bytes  75466.06 bytes/sec
total size is 10693902652  speedup is 8588.17

Update with no files to upload

localhost:~ local-account$ rsync -a -e  'ssh -p 22 ' --progress -P /Users/local-account/folder-to-backup [email protected]:/backup/the-computer/
[email protected]'s password: 
building file list ... 
51354 files to consider

sent 1198459 bytes  received 20 bytes  82653.72 bytes/sec
total size is 10693902652  speedup is 8922.90

Backup is easy..

rsync -a -e  'ssh -p 22 ' --progress -P /Users/local-account/folder-to-backup [email protected]:/backup/the-computer/

If you want incremental and full backups try Duplicity.

Hope this helps.

Donate and make this blog better

Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.7 Duplicity

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

Advice

Setup a Certification Authority Authorization (CAA) DNS record(s) to prevent https cert issue/misuse

DNSSEC

Testing DNSSEC

Testing CAA (on your website)

Adding DNS CAA records at Namecheap

Test CAA Records

Warning

I am moving away from Apple hardware

Using Platforma Web Wireframe Kit to build a website (prototype)

Building faster web apps with google tools and exceed user expectations

Infographic: So you have an idea for an app

Quick guide to using Adobe XD CC to design a prototype iOS app.

How to use Sublime Text editor locally to edit code files on a remote server via SSH

Run an Ubuntu VM system audit with Lynis

Installing and managing WordPress with WP-CLI from the command line on Ubuntu

Popular

Security

Code

Tech

Wordpress

General