How to detect in progress downloads?

[spooknik]

What is the problem you are having with rclone?

I would like to preface this by saying RClone works great, but I would like to run a clean up script in my downloads folder once per day, but I do not wish to delete anything that is currently being downloaded.

Is it possible for Rclone to end file names with something while they are in progress or maybe there is another way to detect if Rclone is accessing the file?

Run the command 'rclone version' and share the full output of the command.

rclone v1.56.1
- os/version: debian 11.5 (64 bit)
- os/kernel: 5.15.107+truenas (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.16.8
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

FTP

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone move --progress --error-on-no-transfer –log-file=log.txt seedbox:/downloads/complete  /storage/media/_sync/temp_download

The rclone config contents with secrets removed.

--------------------
[seedbox]
type = ftp
host = ***REDACTED***
user = spooknik
pass = *** ENCRYPTED ***
tls = true
explicit_tls = false

A log from the command with the -vv flag

Paste log here

[kapitainsky]

Update your rclone version - you use very old one.

@spooknik

Is it possible for Rclone to end file names with something while they are in progress or maybe there is another way to detect if Rclone is accessing the file?

What exactly you want to achieve here?

You run rclone move - if file is successfully moved it will be deleted from source.

Also I see you run it on TrueNAS - you could always utilise snapshots.. but difficult to advice more not knowing what is the final goal.

[spooknik]

@kapitainsky

Update your rclone version - you use very old one.

It's the shipped version TrueNAS Scale comes unfortunately. So i'll update when it's possible from upstream

@kapitainsky

What exactly you want to achieve here?

I have a script that calls Rclone move if there are new files on my Seedbox to download to my local TrueNAS install. If there is it downloads them. Sonarr then picks up on the local downloaded files, and organizes the files by copying them to a folder, etc. Sonnar cannot delete the downloaded files, so they just sit there and take up space in the downloads folder.

Ideally I'd like to run a script periodically that purges the downloads folder, but I want to make sure rclone is not in the middle of a download during a purge.

(roundabout way, maybe, but there is literally no better way to do this)

[kapitainsky]

@spooknik

It's the shipped version TrueNAS Scale comes unfortunately.

Your system your problem:) you do not have to use what it is shipped with. If you like retro computing you will have to deal with a lot of issues on your own.

It sounds at the end like problem with this software called Sonarr - not with rclone... maybe there are some people here who know it. Otherwise you should maybe try Sonarr community forum?

[spooknik]

@kapitainsky

Your system your problem:) you do not have to use what it is shipped with. If you like retro computing you will have to deal with a lot of issues on your own.

True enough! I will update Rclone if it's necessary, at the moment the version installed is working just fine.

@kapitainsky

It sounds at the end like problem with this software called Sonarr - not with rclone... maybe there are some people here who know it. Otherwise you should maybe try Sonarr community forum?

Yes, I had a similar thought and already asked on the Sonarr forums. The best advice was to make a purge script like I mentioned above.

In general I don't like making scripts that delete things blindly without some checking beforehand, so that's why I was asking if somehow it's possible to know if rclone is currently downloading a file.

Like when you download from Chrome browser it puts a .crdownload extension on the file before its finished. Then in the script you can just make a condition to exclude all files names with that extension.

[darthShadow (Anagh Kumar Baranwal)]

If you are copying files to a local filesystem then rclone will use the .partial extension for any in-progress files (added in 1.63.0). You can exclude any files with the .partial extension from your purge script and that should skip any in-progress files.

That said, it may be easier to run a custom script via the connection feature in sonarr that deletes only the file that has already been imported.

[spooknik]

Good to know Rclone has that feature now, I could see it being useful in a few cases.

Yea, I will try to make a custom script in sonarr that runs on import. I am not sure how to selectively delete the imported files. It's outside the scope of Rclone, so I will ask over on the Sonarr forums.