Ok, this is the dumbest thing ever but I'm going to note it here for posterity in case someone else runs into something similar.
I did not want to sync my entire zpool up to Backblaze, mainly because the upload speed at my house is poo. So I created /srv/remote/
as the root of a hierarchy for bind mounts. That way I could just use /srv/remote/
as the source for my offsite backups and get just the most important stuff.
Originally when I first set all this up, I just used the live file system to get my backup script in to production quicker. Most of the data was static but I did have a few daemons updating files underneath the backup script which would occasionally cause size/hash errors for rclone. I put this issue on the TODO list and sat on it for a while.
Months later, I didn't think too much about it and switched to using my nightly ZFS snapshot as the source rather than the live file system. As using a snapshot would mean no more size/hash mismatch errors after a transfer. Well...the bind mounts aren't mounted to a bunch of empty directories in the snapshot, they are only mounted on the live file system. So of course rclone isn't going to copy data that isn't there.
The kicker is that the one file rclone did continue looking at for me, the README.txt in my OP logs, that is where I documented how I set all this up based on bind mounts. I just didn't read my own README.txt before updating my backup script.
The lesson here is:
- Don't try to tackle a software change early in the morning, just sleep on it.
- Read your own README.txt before changing anything.
sigh