--min-age filter not working

What is the problem you are having with rclone?

rclone not obeying --min-age 6h filter.
File created 25 August 2019, 13:56

What is your rclone version (output from rclone version)

rclone v1.49.3

  • os/arch: darwin/amd64

  • go version: go1.12.9

Which OS you are using and how many bits (eg Windows 7, 64 bit)

OSX Catalina

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

while :; do rclone move ./ kinvaratrail: --min-age 6h -P --transfers 2; sleep 1h; done

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

Log

2019/10/25 19:38:28 DEBUG : --min-age 6h0m0s to 2019-10-25 13:38:28.582586 -0500 CDT m=-21599.978602170

2019/10/25 19:38:28 DEBUG : rclone: Version "v1.49.3" starting with parameters ["rclone" "move" "./" "kinvaradrive:" "--min-age" "6h" "--transfers" "2" "-vv"]

2019/10/25 19:38:28 DEBUG : Using config file from "/Users/mz/.config/rclone/rclone.conf"

2019/10/25 19:38:29 INFO : Google drive root '': Waiting for checks to finish

2019/10/25 19:38:29 INFO : Google drive root '': Waiting for transfers to finish

2019/10/25 19:38:37 DEBUG : Pre-Lic_Imm-Checklist.pdf: MD5 = b29d9d62b3c94c9f1e5c585addc3a280 OK

2019/10/25 19:38:37 INFO : Pre-Lic_Imm-Checklist.pdf: Copied (new)

2019/10/25 19:38:37 INFO : Pre-Lic_Imm-Checklist.pdf: Deleted

I ran another test, with a fresh screenshot of the directory, and a copy of the file that transferred earlier, but shouldn't have. This time I set the age to 24h. The screenshot was correctly excluded, but the duplicate of the file wasn't excluded, and ended up in the move.

Log

while :; do rclone move ./ kinvaradrive: --min-age 24h --transfers 2 -vv; sleep 3600; done

2019/10/25 19:44:10 DEBUG : --min-age 1d to 2019-10-24 19:44:10.803687 -0500 CDT m=-86399.976742008

2019/10/25 19:44:10 DEBUG : rclone: Version "v1.49.3" starting with parameters ["rclone" "move" "./" "kinvaradrive:" "--min-age" "24h" "--transfers" "2" "-vv"]

2019/10/25 19:44:10 DEBUG : Using config file from "/Users/mz/.config/rclone/rclone.conf"

2019/10/25 19:44:10 DEBUG : Screen Shot 2019-10-25 at 7.43.35 PM.png: Excluded

2019/10/25 19:44:11 INFO : Google drive root '': Waiting for checks to finish

2019/10/25 19:44:11 INFO : Google drive root '': Waiting for transfers to finish

2019/10/25 19:44:20 DEBUG : Pre-Lic_Imm-Checklist copy.pdf: MD5 = b29d9d62b3c94c9f1e5c585addc3a280 OK

2019/10/25 19:44:20 INFO : Pre-Lic_Imm-Checklist copy.pdf: Copied (new)

2019/10/25 19:44:20 INFO : Pre-Lic_Imm-Checklist copy.pdf: Deleted

EDIT: Adding a stat.

$ stat Pre-Lic_Imm-Checklist\ copy\ 2.pdf 
16777220 4087997 -rw-r--r-- 1 mz staff 0 560808 "Aug 25 13:56:28 2019" "Aug 25 
13:56:28 2019" "Oct 25 19:51:55 2019" "Aug 25 13:56:28 2019" 4096 1096 0 Pre-Lic_Imm-Checklist copy.pdf

You've told rclone to use a minimum age here, so the file Pre-Lic_Imm-Checklist\ copy\ 2.pdf which was last modified on " 25 August 2019, 13:56" is older than that, so it gets transferred.

At least that is what I understand. Is that correct? Maybe you wanted --max-age ?

This could be a problem of bad arithmetic on my part. Maybe I'm just misunderstanding how rclone is working, or I'm missing something else. So correct any flaws in my thinking. Here's what I've got:

Pre-Lic_Imm-Checklist\ copy\ 2.pdf was created at 13:56. According to the debug log, it's going back 6 hours from now, and any file created prior to that time shall be included in the move. Is that right?
2019/10/25 19:38:28 DEBUG : --min-age 6h0m0s to 2019-10-25 13:38:28.582586 -0500 CDT m=-21599.978602170

That's at least how I'm interpreting that line. If that's correct, then, now - 6h = 13:38 and rclone should transfer any file created prior to 13:38. Since my file was created at 13:56, it should be excluded, I would think.

Am I misunderstanding, or miscalculating? Maybe I do want --max-age instead of --min-age? I was hesitant in posting this in the first place, suspecting error on my part rather than a bug. After double checking, I can't see what I'm doing wrong- hence I filed it as a bug.

Just to clarify what I'm trying to do... I have a folder that occasionally has files dropped into it at times. I want those files to be moved to GDrive only if they are older than 6 hours.
In fact, ideally they would only be moved 6h after having been copied to the folder, but I couldn't find a flag that handled anything like that. (I don't even see where OSX is recording that. Files that were created 9 months ago and last modified 2 months ago will always be more than 6h in 'age' even if I did just copy it to that folder 5 seconds ago- no timestamps are being updated that I can see)

Rclone uses the modified time for its --min-age calculations, not the created time.

If you do rclone lsl /path/to/Pre-Lic_Imm-Checklist\ copy\ 2.pdf then rclone will show you what it things the modified time is. That will probably shed some light.

--min-age is the correct flag for that :slight_smile:

There isn't a flag for that and it isn't possible to work that out from just looking at the file.

Wouldn't you be able to tell from the last-access time?
A --min-access-age would be really useful.
Am I just not seeing a technical limitation here? Doesn't the cache use Atime ?

Modify time was right around the same as creation time. But at any rate, if the file was created less than 6h ago, wouldn't modtime also be < 6h, according to rclone?

Yeah, I didn't expect there to be one.

Can you do

rclone lsl /path/to/Pre-Lic_Imm-Checklist\ copy\ 2.pdf

so we can see what rclone things the modified date is?