Experience uploading .sparsebundle (macOS) via rclone?

Help and Support
1

Hi, wondering if anybody has any experience with copying macOS sparsebundles (TimeMachine backup) to a cloud storage (I use Google Drive) via rclone?

A sparsebundle is in practice a directory with many smaller files:

Instead of a single big file, a sparse bundle is a bundle (directory) containing a number of files called bands, each on the order of 8 MB in size. This means even though to the end user the sparse bundle appears as a single file, it is composed of smaller files. As of Mac OS X 10.8, the bands are 8 MiB (8 × 10242 byte) each. When the content of the image changes, one or more band files is changed, created, or deleted. This allows backup software (such as Time Machine) to operate more efficiently.

Thanks.

1 Like
2

Rclone will deal happily with a directory with lots of files within - not sure exactly how a sparsebundle will appear to rclone though… rclone ls /path/to/sparsebundle will tell you though!

Note that rclone doesn’t preserve resource forks on the mac yet either :frowning:

1 Like
3

Thank you. I’ll give it a go and see how it goes.

4

Did it work? I have been meaning to do exactly this for a while with a big sparse bundle. But my upload speed is too slow to sacrifice backup time.

5

Have not gotten that far yet. I guess you could do a test with a smaller one first.

6

I’d be interested to hear how it works out. I considered this kind of backup, but was worried about a few things:

Since, as you describe, a sparse bundle is really a filesystem, changes in a single file (within the bundle’s filesystem) might result in any number of changes in the underlying bands. There’s not going to be a strict correspondence between actual modified files and the amount of incremental change to the sparse bundle’s bands.

The default band size, being 8Mib, was likely chosen to optimize performance for a device mounted locally. Apple expects Time Machine backups to be on the local network. So this becomes your smallest amount of change (rclone doesn’t upload parts of files yet, does it?) for uploads. This might need tuning.

As a very simple example, a laptop I have backing up to Time Machine has a 112 GB drive. In the last day, over 1400 bands of its TM backup have been modified (even though I’ve not done a thing on that machine). So that would be a incremental backup of almost 11GB. (Perhaps in your use case it might be acceptable, but it’s overkill for me.)

Also, what happens if part of an upload fails? For example, if 30 bands need uploading, and one fails, what happens when you restore from that incomplete backup? It would be tricky to figure out how that translates to the actual files—would the sparse bundle, after being mounted, know that the data was wrong? Or would you be stuck with a smattering of incorrect files (& possibly directories) and no way of knowing which?