Exclude Dependencies from Time Machine Backups

For as long as I’ve been a developer, I’ve been working (almost) exclusively on Macs. I don’t have any particular allegiance to Apple or anything — in fact, many of the tools I use are either cross-platform or are running in virtual machines or containers — but I’m comfortable with the hardware and don’t have enough complaints to motivate me to switch.

If you’ve used a Mac in the last decade or so, you’ve likely been prompted to configure Time Machine, macOS’ built-in automated backup solution; simply connect your backup disk (or use certain network attached storage devices) and Time Machine will automatically make incremental backups of your machine. In the event that your computer’s lost/stolen, its hard drive is corrupted, or you simply deleted that super important file, Time Machine makes it easy to restore your computer’s previously healthy state.

(Obligatory time travel reference)

Time Machine v developers

Where Time Machine is less convenient is in the case of developers: modern development practices often rely on dependency management tools (e.g. Composer, npm, etc.) to pull in third-party dependencies. Instead of including full copies of external libraries, developers can say “my application relies on package N at version X.Y.Z”, and the dependency manager can download the necessary code as a build step. This is great for keeping third-party assets both versioned and out of version control, but for the developer working on multiple projects it poses a bit of a problem: you end up with a ton of project dependencies on your machine!

When I left 10up last year, I did an inventory of all the old projects on my machine; just within Composer’s vendor/ directories and npm’s node_modules/directories, I had gigabytes of leftover files hanging around; files that, if I needed to, I could restore easily by running their respective dependency managers’ install commands. Worse yet, every time I ran a Time Machine backup, these dependency files were synchronized, taking up space that could be used for more important things (read: animated gifs and pictures of my daughter).

I went digging to see if I could have Time Machine ignore files and directories matching a specific pattern (a la .gitignore), but Time Machine only lets you specify exact file paths to exclude. Figuring that to be far too tedious for my lazy self, I went looking for a way to programmatically add items to Time Machine’s exclusion list. Eventually, I stumbled upon this StackOverflow thread, which turned me onto macOS’ tmutil command:

tmutil provides methods of controlling and interacting with Time Machine, as well as examining and manipulating Time Machine backups. Common abilities include restoring data from backups, editing exclusions, and comparing backups.

Several, but not all, verbs require root privileges.

With tmutil, I’m able to programmatically set Time Machine exclusions, a la:

Automatically exclude project dependencies from Time Machine backups

With tmutil in-hand, it wasn’t much of a stretch to write a shell script to automatically find all the common project dependency directories on my machine and exclude them from Time Machine’s backups. The result is Asimov, which is now available on GitHub.

Asimov works by iterating through known dependency file naming schemes and — after verifying the dependency declaration is present — marks the directory for exclusion from Time Machine backups; instead of automatically excluding all instances of node_modules/, Asimov will only exclude those that sit adjacent to a package.json file (likewise for vendor/ and composer.json).

To install Asimov and automatically schedule it to run daily, you can use the following one-liner:

The install script (only available in the develop branch, at the time of this writing) will automatically register Asimov to run every 24 hours, ensuring your project dependencies are unlikely to get caught in Time Machine.

I’d love to see more development dependency patterns to match (being primarily a PHP developer, I’m really only familiar with Composer and npm, though I’m sure there are plenty more). My hope is that Asimov becomes a must-install script for any and all developers working on macOS. If you’d like to get involved with the ongoing development of Asimov, please visit the GitHub repository!