Min-age flag not being honoured

What is the problem you are having with rclone?

Min-age flag is not being honoured by rclone

Run the command 'rclone version' and share the full output of the command.

rclone v1.61.1

  • os/version: ubuntu 22.04 (64 bit)
  • os/kernel: 5.15.0-53-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.19.4
  • go/linking: static
  • go/tags: none

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
-->

Which cloud storage system are you using? (eg Google Drive)

google drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

clone -v --drive-impersonate email@email.com -min-age 15m --no-check-certificate --check-first --no-traverse --drive-chunk-size 256M --buffer-size 512M --delete-after --min-size 1M --stats 30s --transfers=24 --multi-thread-streams 8 --drive-stop-on-upload-limit move /ssd/localgdrive/ googleupload001decrypted:Data --exclude .@*

The rclone config contents with secrets removed.

Not needed

A log from the command with the -vv flag

Not appicable

You'd really want to share the rclone.conf and a debug log as there's nothing to look at to help you out.

That's why we ask for that stuff.

hello and welcome to the forum,

best way to test a filter is rclone ls on a single source file.
something like this:
rclone ls /ssd/localgdrive/filename.ext --min-age 15m -vv --drive-impersonate email@email.com
and post the full output.

2023/05/20 16:36:03 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "ls" "/ssd/localgdrive/TVShows/Grey's Anatomy/Season 19/Grey's Anatomy - S19E08 - All Star - WEBDL-1080p - NTb.mkv" "--min-age" "15m" "-vv" "--drive-impersonate" "beth@email.com"]
2023/05/20 16:36:03 DEBUG : Creating backend with remote "/ssd/localgdrive/TVShows/Grey's Anatomy/Season 19/Grey's Anatomy - S19E08 - All Star - WEBDL-1080p - NTb.mkv"
2023/05/20 16:36:03 DEBUG : Using config file from "/home/plex/.config/rclone/rclone.conf"
2023/05/20 16:36:03 DEBUG : fs cache: adding new entry for parent of "/ssd/localgdrive/TVShows/Grey's Anatomy/Season 19/Grey's Anatomy - S19E08 - All Star - WEBDL-1080p - NTb.mkv", "/ssd/localgdrive/TVShows/Grey's Anatomy/Season 19"
2023/05/20 16:36:03 can't limit to single files when using filters: /ssd/localgdrive/TVShows/Grey's Anatomy/Season 19/Grey's Anatomy - S19E08 - All Star - WEBDL-1080p - NTb.mkv

After running

rclone ls "/ssd/localgdrive/TVShows/Grey's Anatomy/Season 19/Grey's Anatomy - S19E08 - All Star - WEBDL-1080p - NTb.mkv" --min-age 15m -vv --drive-impersonate beth@email.com
rclone ls "/ssd/localgdrive/TVShows/Grey's Anatomy/Season 19" --include="Grey's Anatomy - S19E08 - All Star - WEBDL-1080p - NTb.mkv" --min-age 15m -vv --drive-impersonate beth@email.com

2023/05/20 16:42:18 DEBUG : --min-age 15m0s to 2023-05-20 16:27:18.371401851 +0000 UTC m=-899.983576415
2023/05/20 16:42:18 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "ls" "/ssd/localgdrive/TVShows/Grey's Anatomy/Season 19" "--include=Grey's Anatomy - S19E08 - All Star - WEBDL-1080p - NTb.mkv" "--min-age" "15m" "-vv" "--drive-impersonate" "beth@jumbo.stream"]
2023/05/20 16:42:18 DEBUG : Creating backend with remote "/ssd/localgdrive/TVShows/Grey's Anatomy/Season 19"
2023/05/20 16:42:18 DEBUG : Using config file from "/home/plex/.config/rclone/rclone.conf"
3112167957 Grey's Anatomy - S19E08 - All Star - WEBDL-1080p - NTb.mkv
2023/05/20 16:42:18 DEBUG : 2 go routines active

and what is the problem with that output?

I use this command to upload

rclone -v --drive-impersonate beth@jumbo.stream --min-age 15m --no-check-certificate --check-first --no-traverse --drive-chunk-size 256M --buffer-size 512M --delete-after --min-size 1M --stats 30s --transfers=24 --multi-thread-streams 8 --drive-stop-on-upload-limit move /ssd/localgdrive/ googleupload001decrypted:Data --exclude .@*

But it still upload things that have been imported to the disk before the 15minute mark. Shouldn't it only transfer files that have been on disk for 15 minutes or longer?

i am not asking about upload things.
i am asking only about the output of the rclone ls command.
is that output correct or not?

if the output is not correct, then post the output of
rclone lsl /path/to/filename.ext" -vv

I don't know if its correct or not, that is why I'm asking here for help?

The min-age isn't working, I don't know any troubleshooting steps as to find out why, you gave me a command to try and I posted the output, I don't really understand what it means.

Sorry

here is a simple example, two files.
newer.than.15min.txt - modtime is newer than 15 minutes
older.than.15min.txt - modtime is older than 15 minutes

the first command lists all files, with modtimes
the second command list files with modtimes older than 15 minutes

rclone lsl /home/user01/rclone/scripts/test
        1 2023-05-20 13:00:46.352207400 newer.than.15min.txt
        1 2022-05-20 12:58:25.029000000 older.than.15min.txt

rclone lsl /home/user01/rclone/scripts/test --min-age=15m
        1 2022-05-20 12:58:25.029000000 older.than.15min.txt

Can you show example what is not working? Not just description.

For example:

File on your system which is less than 15 min old (do ls/dir - whatever is your OS)

Then do rclone ls -vv ... -min-age 15m to show that rclone does not ignore this file and list it.

the OP already posted an example file older than 15 minutes.
trying to be sure the OP understands what --min-age.

not sure what you mean about imported and before?

correct, if the modtime of the source file is older than 15 minutes, rclone will transfer it.
based on your rclone ls command and my example, that is what rclone is doing.

Okay.

This is not what rclone is doing.

Sonarr imports a file to disk, I start an upload immediately after import, it transfers immediately. This should not be the intended behaviour?

if that is not correct, then

@Jason_Lucas, that is the next step.
i provided an example on how to do that.

you need to find a file that shows that rclone does not honor --min-age.
after sonarr has imported the file, but before rclone move

run the rclone lsl commands on that file and post the full debug output.
like my example up above, something like this

rclone lsl /path/to/source --include=file.ext -vv
rclone lsl /path/to/source --include=file.ext min-age=15m -vv

Apologies, here's one file that just imported

2023/05/20 23:16:22 DEBUG : --min-age 15m0s to 2023-05-20 23:01:22.667765871 +0000 UTC m=-899.978300206
2023/05/20 23:16:22 DEBUG : rclone: Version "v1.61.1" starting with parameters ["rclone" "lsl" "/ssd/localgdrive/TVShows/Fire Country tvdb-420063/Season 01" "--include=Fire Country - S01E12 - Two Pink Lines - WEBDL-1080p - NTb.mkv" "--min-age" "15m" "-vv"]
2023/05/20 23:16:22 DEBUG : Creating backend with remote "/ssd/localgdrive/TVShows/Fire Country tvdb-420063/Season 01"
2023/05/20 23:16:22 DEBUG : Using config file from "/home/plex/.config/rclone/rclone.conf"
2023/05/20 23:16:22 DEBUG : Fire Country - S01E12 - Two Pink Lines - WEBDL-1080p - NTb.mkv: Excluded
2023/05/20 23:16:22 DEBUG : 2 go routines active

Here it states it is excluded. However, when I run my command to upload;

rclone -v --drive-impersonate beth@jumbo.stream --min-age 15m --no-check-certificate --check-first --no-traverse --drive-chunk-size 256M --buffer-size 512M --delete-after --min-size 1M --stats 30s --transfers=24 --multi-thread-streams 8 --drive-stop-on-upload-limit move /ssd/localgdrive/ googleupload001decrypted:Data --exclude .@*

The very same "Fire Country - S01E12 - Two Pink Lines - WEBDL-1080p - NTb.mkv" file is uploaded.

ok, that is good but i should have suggested.
use --dry-run with rclone move so rclone will not upload the file.

Ok this "testing" is going nowhere. Clearly.

Good thing is I know what the problem is.

rclone works as expected and fully respects --min-age flag (unless somebody can provide example to the contrary). But rclone does not have magic powers (until ChatGPT will be integrated one day) to know how long time ago file was modified. It uses old fashioned file's date/time attributes - normally they correspond to real file system actions on given file unless 3rd party software modifies them.

And this is exactly the case here. Sonarr (torrent client??) set date/time not to the moment where file is saved but to the original film air date. Such a funky feature of this particular program.

@Jason_Lucas - if you do not like this Sonarr behaviour I suggest you change its settings (if possible) or seek help among Sonarr community.

And here similar issue:

@Jason_Lucas - maybe it will give you some ideas how to fix it. It is not rclone issue but your software trickery/feature:)

I think this is the source of the confusion... --min-age looks at the modification time of the file, which will be the same as the modification time of the file when it was uploaded not the time the file was uploaded.