Sync folders from different drives to same remote

[Lombra]

Hi,

I'm looking to sync folders A and B from drive X:, as well as folder C and files in the root from drive Y:, to remote backblaze.

For now, I am syncing only drive X. If I understand correctly, it is not possible to include drive Y in the same command.

rclone sync X: --include A/** --include B/** backblaze:

I tried syncing drive Y to the same folder.

rclone sync Y: --include /* --include C/** backblaze:

I thought that this would delete backblaze:\A and backblaze:\B, because it would sync Y: -> backblaze: while including only the listed paths. It did not do that, which is great and just what I need. So I gather that the --include flag calculates the sync operations separately per included path, rather than the whole source path, with --include acting as a simple filter.

I just want to make sure that I have understood this all correctly, and that this is the intended behavior, so that I can expect to accomplish what I want by running those two commands.

[asdffdsa (jojothehumanmonkey)]

hello and welcome ot the forum,

i can think of a few ways.

[union]
type = union
upstreams = x:\:ro y:\:ro

rclone sync union: --include="/{A,B,C}/**" backblaze: --dry-run

[ncw (Nick Craig-Wood)]

@Lombra

So I gather that the --include flag calculates the sync operations separately per included path, rather than the whole source path, with --include acting as a simple filter.

Rclone won't delete stuff that is excluded from the sync. So if you've got --include A/** it won't delete anything outside of A. (Unless you include the --delete-excluded flag of course!)

@Lombra

I thought that this would delete backblaze:\A and backblaze:\B, because it would sync Y: -> backblaze: while including only the listed paths.

The reason this works is that you've written --include /* which can only match files in the root. If you'd have written --include /** then it would have removed A and B.

@Lombra

I just want to make sure that I have understood this all correctly, and that this is the intended behavior, so that I can expect to accomplish what I want by running those two commands.

Yes it looks good If you ever create a file called A or B in the root of Y: then there will be trouble though.

To work around this you'd exclude A and B from that sync. You need to use --filter for this as you can't mix --include and --exclude on one line. So something like

--filter "- A/**"
--filter "- B/**"
--filter "+ /*"
--filter "+ C/**"
--filter "- **"

At some point this becomes more convenient to put a in a --filter-from file.

[Lombra]

Great, thanks!

@ncw

If you ever create a file called A or B in the root of Y: then there will be trouble though.

Right. Makes sense.

Unions look interesting. Hadn't seen those. Consolidating into a single command would be great, but I have some conflicting symlinks, such as X:\C -> Y:\C. I understand these are resolved using policies, but I'm not entirely sure what would happen in this case. I suspect only the symlink would be synced. Is it dependent on your symlink parameters? I need to sync while preserving symlinks deeper in the directories.

[asdffdsa (jojothehumanmonkey)]

@Lombra

I need to sync while preserving symlinks deeper in the directories.

maybe another variation

[combine]
type = combine
upstreams = X=X:\ Y=Y:\

rclone copy combine: backblaze: --links --include=/X/{A,B}/** --include=/Y/C/** -vv --dry-run

[Lombra]

Thank you. It would be neat to have everything in a single union path, but I think I will go with combine and specify exactly what I need.

[asdffdsa (jojothehumanmonkey)]

@Lombra

It would be neat to have everything in a single union path

might be possible, to use a combination of combine and union.
as i do not use either type of remote, cannot help with that.