Rclone not exiting properly - Google team drive

Help and Support
1

rclone seems to not be exiting properly, it just hangs at the end of the files transferred and seems to be downloading at a reduced rate. This has the issue of causing my script to get stuck and not move on to the next service account

rclone v1.57.0

  • os/version: debian 11.2 (64 bit)
  • os/kernel: 5.10.0-11-amd64 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.17.2
  • go/linking: static
  • go/tags: none

Info: I am using service accounts to download from an encrypted google team drive

Commmand used:

/usr/bin/rclone copy dst335_crypt:storage /mnt/local/storage --config=/home/username/.config/rclone/rclone.conf --drive-server-side-across-configs -vv --include /{K,L,M,N,O,P,Q}/** --min-age 2m --tpslimit 6 --transfers 5 --drive-chunk-size 512M --drive-acknowledge-abuse --max-transfer=550G --cutoff-mode=soft --log-file=/home/username/logs/rclone_log.txt

There's a few thousand lines in the log so that's available on request, but the relevant part that I could find was at the end:

pastebin with some context (not full log):
https://pastebin.com/raw/T142V4SG

2022/03/04 19:31:03 INFO  :
Transferred:      588.948 GiB / 9.857 TiB, 1%, 4.666 MiB/s, ETA 36w3d15h15m5s
Errors:                 1 (no need to retry)
Checks:               532 / 532, 100%
Transferred:           60 / 10067, 1%
Elapsed time:      31m1.9s

It repeatedly slows down to a crawl for some reason, and I'm not seeing any new file transfers being started or any files in current progress. I suspect it might have something to with the cutoff-mode, and I got the same with cutoff-mode cautious as well.

2

I think this might be a bug related to cutoff mode which is fixed in the latest beta can you give that a go?

3

Thank you, that seems to be working so far.

1 Like
4

Never mind, it happened again tonight. Previously I was able to recreate the issue everytime. It was constantly hanging.

However, this time, it did not occur with the service accounts, but a normal gsuite account with its own client ID/secret. The service accounts have so far not hung, and the for the individual gsuite account, I am unable to recreate it consistently.

For reference

rclone v1.58.0-beta.6018.ec72432ce
- os/version: debian 11.2 (64 bit)
- os/kernel: 5.10.0-11-amd64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17.7
- go/linking: static
- go/tags: none

The command is the same, except for the remote name.

5

If you can get it to happen again, a log with -vv would be very useful as would killing the process with kill -QUIT which will dump a backtrace showing exactly what rclone is up to.