Does --max-age only check files of that age?

Feature
1

basic question about how rclone processes files when using this filter. the way i'm using it:

rclone copy path/to/local remote:destination --max-age 1h

question:
will rclone check for excluded files on the remote side? the reason i ask is, the --vv filter says the files are excluded, but the operation takes the same amount of time as a full sync does. the output log in the end also does not show the amount of files processed. just the amount of files transferred. this even when using --verbose and --progress.

i'm asking because this is supposed to be a way to reduce I/O on large operations. that's what i'm after here.

1 Like
2

It is just yet another filter - for any filter to work you have to list all objects first.

It does for specific situations only. For example when transferring from local to remote with additional --no-traverse flag. This way you avoid listing and checking all remotes objects and only transfer files changed locally recently (since last full copy maybe).

--max-age definitely is not a flag helping all large operations.

1 Like
3

perfect. using --no-traverse i was able to copy all new files on a 400,000 file job in mere seconds.

there is a job doing a full 'sync' on this source/dest every night. i'm just wanting something that i can use to run a quick copy of all new files that have changed during this day. this is important when i want to run a backup quickly before doing maintenance work, but i don't want to wait a few hours for all the files to get run through. in that scenario, doing deletes on the remote isn't quite as important.

4

Which is exactly the use case when it can make massive difference (together with --no-traverse).

Effectively you copy some files blindly (without checking destination). It would be very slow when done for all files (with most already existing) but by using --max-age pretty much you make sure that all transfers are OK

1 Like