Having achieved something akin to success getting drag-and-drop file transfers working in Reconnect on day 19 of my December Adventure, I hit a bit of a brick wall—life happened (as it often does), and then I found myself distracted by some of the more mundane aspects of technology. Still, yesterday’s tinkerings feel like they might make for an intriguing, albeit not Psion-related, write-up and I didn’t want to leave my journal unfinished at day 19.

Tracking File Changes

Since moving away from centralized subscription-based infrastructure a few years ago, I’ve been happily using Syncthing for synchronizing files between my devices. It’s far from perfect—the user interface is clunky and setup is nuanced—but once configured, it feels robust and keeps my files in sync across macOS, Linux, TrueNAS, and iOS¹.

Since I like to keep a few machines in the daily rotation, I have a fairly involved Syncthing setup: I have two always-on server nodes (local and remote), three portable computers, and a smartphone. It looks something like this:

• iPhone (Kotoko) — iCloud Backup (Excluding Syncthing Folders)
• MacBook Pro (Chii) — Backblaze Backup
• Pocket Reform (Sumomo)
• TrueNAS Server (Zima) — Remote Mirror — Backblaze B2 Backup (Introducer, Auto-Accept)
• Raspberry Pi Server (Dita) — Local Mirror (Receive-Only)
• Little Luggable

The problem with Syncthing is its primary feature: changes propagate to all devices. This means you’re exposed to problems on every device and, a couple of days ago, I noticed a directory in one of my Syncthing folders was unexpectedly empty. Now, I don’t remember if it was empty (it’s quite possible I started sorting something out and never finished), but I suspect it wasn’t. That means a file has silently been deleted. This could be a bug in Syncthing itself, or an issue on one of my computers that Syncthing has then propagated.

Synchronization amplifies these kinds of failures by exposing you to an increased number of placs where things can go wrong. Without even considering the potential failures of synchronization itself. The design of of Syncthing also means that, by default, there’s no centralized place to go to audit changes. But these issues are not limited to synchronzied files—if your OS, file system, or hardware is unreliable such failures can still occur on a standalone device.

I have explicit backup solutions for a number of the computers in this setup: the iPhone backups up to iCloud, my MacBook Pro to Backblaze, and my TrueNAS Server to Backblaze B2. But these backups aren’t very useful if you don’t know that you need them—if infrequently used files slowly disappear one at a time over the course of years.

Tracking Changes

Spurred on by my discovery, I set about seeing what options were out there to help me manage my files and detect upexpected changes early. I’ve always liked Dropbox’s approach to sending notifications of unexpected events like changes to large numbers of files, and would love something like that for my Syncthing solution. My hope is that something similar should let me quickly restore changed or deleted files from a backup should the worst happen.

Since all my the files I care about are syncrhonized using Syncthing, my idea is to run a task that tracks the files on my always-on, backed up, TrueNAS server, Zima. TrueNAS uses ZFS so I don’t expect any failures to originate on that server, but it feels like a great place to notice unwanted changes that have propagated through synchronization.

Initial Experiments

Not wanting to reintent the wheel, I searched for off-the-shelf alternatives. The available tools seem to fall into two camps: integrity checking and access auditing. Tools like Tripwire focus on integrity checks for detecting compromised servers and are set up to track deviations from a known good state instead of an on-going stream of changes. Conversely, auditing tools like auditd offer—for my usecases—overly fine-grained monitoring and (superficially at least) seem to require a host of tools to analyse the logs.

I decided to conduct some experiemtns on my local Raspberry Pi server, Dita. This is set up as a ‘receive only’ Syncthing node so, in theory, it will never push changes to other things, making it a great monitoring deivce. For the time being at least.

This was a great motivation to finally configure my Pi so it can send emails. The process proved embarassingly easy (just awkwardly fiddly like most of Linux) and I wish I’d done it sooner. Edafe Knabe has a great write up over at edafe.de.

As I alluded to earlier, Tripwire, while looking initially hopeful, is really set up to maintain system baselines for security reasons and reporting on any file system deviations. It requires an admin user to essentially accept any changes and reinitialize Tripwire using a security key every time changes are detected, otherwise it will continue to report on them. While it would be possible to script this acceptance process, it’s abundantly clear this is working against the tool and adding unnecessary compllexity.

Introducing Reporter

(Who knows if the name will stick.)

Having spent quite a bit of time exploring the existing offerings, I concluded that I want something significantly simpler that does just the handful of things I need. Specifially, I’m afer the following:

• daily emails
• easy to consume summaries
• lists of changes (additions, deletions, renames, and modifications)

Ideally, I’d also like the following:

• respecting Syncthing’s .stignore ignore files
• per-folder policies allowing me to ignore some events (e.g., apppend-only folders which ignore additions but report modifications and deletions); this would be very helpful for my paperwork and documentation folders

Playing around with Tripwire, I found I liked the simple approach taken of just using cron to perform checks. With this in mind, I decided to write a tool which will generate a snapshot of a folder, compare it to a previous snapshot, report any changes, and update the snapshot.

In what might sitll prove to be a poor choice, I decided to write it in Swift—I know the langauge well and want to be able to take advantage of it’s comprehensive concurrencty support. At some point in the future, perhaps I’ll see what would be involved in doing the same thing in Rust, but that’s not for today.

It’s pretty basic right now, but this is a great start.

https://github.com/inseven/reporter

Perhaps Reporter would be useful to you? Let me know. And do let me know if I’ve missed an existing tool that already does everything I want—maintaining less custom software is always a win. I hope to add functionality for detecting file modifications, tracking renames wherever possible, and supporting a wider range of Linux builds in the future but, for now, I’m planning a more chill day 24. The cables have turned up to allow me to try out RISC OS on my Raspberry Pi Zero 2w, so perhaps I’ll give that a go.

As we’re nearing the end of our respective adventures, I encourage you to check out another couple from Psion Discord members: Colin Hoad’s Advent of Beeb Games, and smallsolar’s solar infrastructure write-ups. Both have brought an enjoyable change of pace this month.