--drive-stop-on-upload-limit not working?

What is the problem you are having with rclone?

--drive-stop-on-upload-limit doesn't seem to be working. When my upload hits the 750GB barrier the job doesn't seem to be stopping and no new log entries are written

Run the command 'rclone version' and share the full output of the command.

rclone v1.57.0

  • os/version: ubuntu 20.04 (64 bit)
  • os/kernel: 5.4.0 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.17.2
  • go/linking: static
  • go/tags: none

#####################

I'm running an upload job on a rapidseedbox but when the job hits the 750GB barrier (I think) it's not stopping as expected due to --drive-stop-on-upload-limit.

Also, no new entries get added to the logs which seems odd, so it's probably user error :wink:

Any idea what I'm doing wrong please.

Here's my upload command:

/usr/local/bin/rclone move /home/user/local/seedbox tdrive_boost_upload_vfs:downloads/seedbox --drive-service-account-file=/home/user/rclone/service_accounts/sa_spare_upload11.json --config=/home/user/.config/rclone/rclone.conf --user-agent="external" -vv --order-by modtime,ascending --min-age 3m --drive-chunk-size=256M --transfers=4 --checkers=8 --exclude rutorrent/** --drive-stop-on-upload-limit --delete-empty-src-dirs --bwlimit=100M --log-file=/home/user/rclone/nzbget_log_11.txt

And here's the latest run of the script starting at 16:47 and all good until 17:01 (it's now 17:12) when the transfer stopped (I checked the server) and no more entries added to the logs:

2022/02/24 16:47:04 INFO  : Starting bandwidth limiter at 100Mi Byte/s
2022/02/24 16:47:04 DEBUG : --min-age 3m0s to 2022-02-24 16:44:04.51087992 +0000 UTC m=-179.244988989
2022/02/24 16:47:04 DEBUG : rclone: Version "v1.57.0" starting with parameters ["/usr/local/bin/rclone" "move" "/home/user/local/seedbox" "tdrive_boost_upload_vfs:downloads/seedbox" "--drive-service-account-file=/home/user/rclone/service_accounts/sa_spare_upload11.json" "--config=/home/user/.config/rclone/rclone.conf" "--user-agent=external" "-vv" "--order-by" "modtime,ascending" "--min-age" "3m" "--exclude" "_unpack/**" "--check-first" "--stats-file-name-length" "80" "--drive-chunk-size=256M" "--transfers=4" "--checkers=8" "--exclude" "rutorrent/**" "--drive-stop-on-upload-limit" "--delete-empty-src-dirs" "--bwlimit=100M" "--log-file=/home/user/rclone/nzbget_log_11.txt"]
2022/02/24 16:47:04 DEBUG : Creating backend with remote "/home/user/local/seedbox"
2022/02/24 16:47:04 DEBUG : Using config file from "/home/user/.config/rclone/rclone.conf"
2022/02/24 16:47:04 DEBUG : Creating backend with remote "tdrive_boost_upload_vfs:downloads/seedbox"
2022/02/24 16:47:04 DEBUG : Creating backend with remote "tdrive_boost_upload:crypt/xxxxxxxxxxxxxxxx"
2022/02/24 16:47:04 DEBUG : tdrive_boost_upload: detected overridden config - adding "{N5w4q}" suffix to name
2022/02/24 16:47:07 DEBUG : fs cache: renaming cache item "tdrive_boost_upload:crypt/xxxxxxxxxxxxxxxxx" to be canonical "tdrive_boost_upload{N5w4q}:crypt/xxxxxxxxxxxxxxxxxx"
2022/02/24 16:47:07 DEBUG : fs cache: switching user supplied name "tdrive_boost_upload:crypt/xxxxxxxxxxxxxxxxxxxx" for canonical name "tdrive_boost_upload{N5w4q}:crypt/xxxxxxxxxxxxxxxxxx"
2022/02/24 16:47:07 INFO  : Encrypted drive 'tdrive_boost_upload_vfs:downloads/seedbox': Running all checks before starting transfers
2022/02/24 16:47:07 DEBUG : rutorrent: Excluded
2022/02/24 16:47:07 DEBUG : rutorrent: Excluded
2022/02/24 16:47:09 DEBUG : nzbget/completed/tv/xxxxxxxxxxxxxxxxxx/_unpack: Excluded
2022/02/24 16:47:09 DEBUG : nzbget/completed/tv/xxxxxxxxxxxx/_unpack: Excluded
2022/02/24 16:47:09 DEBUG : nzbget/completed/tv/xxxxxxxxxxx/_unpack: Excluded
2022/02/24 16:47:15 DEBUG : Encrypted drive 'tdrive_boost_upload_vfs:downloads/seedbox': Waiting for checks to finish
2022/02/24 16:47:15 INFO  : Encrypted drive 'tdrive_boost_upload_vfs:downloads/seedbox': Checks finished, now starting transfers
2022/02/24 16:47:15 DEBUG : Encrypted drive 'tdrive_boost_upload_vfs:downloads/seedbox': Waiting for transfers to finish
...
...
...
...
...
2022/02/24 16:47:39 DEBUG : nzbget/completed/tv/xxxxxxxxxx9 OK
2022/02/24 16:47:39 INFO  : nzbget/completed/tv/xxxxxxxxxxxxxx: Copied (new)
2022/02/24 16:47:39 DEBUG : nzbget/completed/tv/xxxxxxxxxxx OK
2022/02/24 16:47:39 INFO  : nzbget/completed/tv/xxxxxxxxx (new)
...
...
...
...
...
...
2022/02/24 17:01:07 INFO  : 
Transferred:   	   48.443 GiB / 1.595 TiB, 3%, 4.035 MiB/s, ETA 4d15h43m5s
Checks:                64 / 68, 94%
Deleted:               32 (files), 0 (dirs)
Renamed:               32
Transferred:           32 / 688, 5%
Elapsed time:      14m2.9s
Checking:

Transferring:
 * nzbget/completed/tv/xxxxxxxx.mkv: 77% /3.186Gi, 1003.458Ki/s, 12m20s
 * nzbget/completed/tv/xxxxxxxxxxxx.mkv: 78% /2.250Gi, 1.007Mi/s, 8m4s
 * nzbget/completed/tv/xxxxxxxxxxx.mkv: 16% /10.458Gi, 1.020Mi/s, 2h26m31s
 * nzbget/completed/tv/xxxxxxxxxxxxx.mkv: 17% /9.439Gi, 1.044Mi/s, 2h7m20s

The logs just end there with no entries or transfer. I assume if I leave until tomorrow it will restart, but I need the script to do other things before then.

Any ideas? Thanks in advance for any help

Hmm, that's a tough one.

I don't see any indication you hit the upload limit yet.

It is transferring files still or does the log just stop? timestamp on the box is what? UTC?

It stops transferring - rapidseedbox have a nice tool to see the transfer quickly - you can see the script starting and stopping in the green blob:

You've made me think though that maybe it's the script or rclone that's crashing on the box. I've restarted it and I'm watching the active processes to see if I spot anything when the transfer stops.

ok you set me off in the right direction. It wasn't rclone - the seedbox was rebooting as I was maxing out resources. My cron job was setup to look for a control file to see if it needed to run again, but the crash was meaning the control file wasn't being removed properly.

I'm now doing this now which seems to be working:

SERVICE="rclone"
if pgrep -x "$SERVICE" >/dev/null
then
    echo "$SERVICE is running"
# don't restart script to upload files
else
    echo "$SERVICE stopped"
    # restart script and rclone job
fi
1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.