How to minimize bandwith w.r.t. renames during sync

Help and Support
1

as an example, let's say that:

  • I have a single big local file in a folder, /folder/bigfile.dat,
  • I have synced that folder to a remote, rclone sync /folder REMOTE:folder,
  • I would like to rename that file, without re-transfering it.

I know I could invoke something like
rclone moveto REMOTE:folder/bigfile.dat REMOTE:folder/newname.dat
but this has the disadvantage that it needs to be done manually.

instead, I am wondering whether there is a way to instruct rclone to do this automatically when using sync. e.g.: rclone sync /folder REMOTE:folder sees that the SAME file has been renamed, and it renames it for me.

is it doable?
thanks!

2

If you use --track-renames, it should do that if I am understanding what you're asking.

1 Like
3

@calisro what does the flag do? I've read the docs, but I'm not sure whether there is something else I'm not seeing.

in other terms, why shouldn't rclone track renames?

4

Tracking renames uses more resources, memory in particular, so it isn't turned on by default.

It isn't excessive so maybe it should be ...

1 Like
5

well, with 16 GB of memory and 160 GB (or more!) of backups, I'd definitely want to avoid transferring files if a rename suffices. :smiley:

thanks for the tip, then. I'll switch the flag on!

6

I got an error:

ERROR : Encrypted drive 'REMOTE:folder': Ignoring --track-renames as the source and destination do not have a common hash

does this rely on hash exclusively?
I'm currently using a union remote of several remotes, and not all support hashes -- am I thus barred from using the --track-renames option?

7

New in 1.52 is that you can add

--track-renames-strategy modtime
8

will try, thanks.

does the --track-renames-strategy modtime flag imply --track-renames?
or do I have to put both?

9

You'll need both.

10

got it.
unfortunately, since I'm using at least a Mega remote in my union, neither option works for me:

ERROR : Encrypted drive 'REMOTE:Folder': Ignoring --track-renames as either the source or destination do not support modtime

:frowning:

11

I'm not sure if this would work through a union but you may be able to set a environment variable to enable the default. and then execute the command.

RCLONE_TRACK_RENAMES=true
RCLONE_TRACK_RENAMES_STRATEGY=modtime

12

that's a great suggestion to avoid exploding length of command-line parameters!
... but, would that influence in any way the fact that the remotes underlying the union might not all have hash or modtime support?
at least, that's what I believe is happening, maybe due to Mega being encrypted even independently of my crypt layer?

13

well, actually, even Mega supports modtime, as I can see from rclone lsl, so I don't know why track-renames with modtimes fails. any thoughts?

14

Its failing because union doesn't support it. I don''t think its possible to specify these types of global flags (yet) for a a remote through the union. I was hoping we could set the default via the environment variable and that would be picked up by the backends but I'm not really sure.

@Max-Sum is there any way to pass these global flags to be used by the backends such as this?

EDIT: The above is incorrect. Its failing because mega doesn't support modtimes.

15

I don't think mega does support modtime. You can read modtimes but you can't set them which you need for the modtime to be accurate.

16

oh, okay, thanks for the clarification.
I'm still a bit confused about the different properties of the various remotes.

so, is this issue due to the fact that the union remote does not support either types of --track-renames?
maybe an explicit error in the debug log could help other users, too?

17

track-renames shouldn't be something attached to filesystem. I can enable --track-renames with union + crypt.

1 Like
18

from what I could gather, it depends on the nature of the remotes underlying the union/crypt.
which ones are you using?
I'm using a union of Box and Mega, and I think the latter is the one at fault for missing features.

19

Yes you'll need all remotes to support modtimes and only Box does, so union will report it doesn't support modtimes.

You can double check this with rclone backend features union: - you'll see the Precision field. If that is very large then modtimes are not supported.

1 Like
20

exactly! I was replying to @Max-Sum.

yet another command that I did not know existed. nice!
still, why doesn't this features command explicitly mention hashes or modtimes?