Stuck at 100% upload on Git Bash

Help and Support
1

What is the problem you are having with rclone?

I wrote a bash script to automatically upload files which I run using Git Bash. Rclone approaches 100% upload without any trouble but at 100% it doesn't finish it and exit, rather it stays there at 0/s

What is your rclone version (output from rclone version)

rclone v1.55.0
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.16.2
  • go/linking: dynamic
  • go/tags: cmount

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Win 10 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive (Shared)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone --progress --drive-chunk-size=128M --log-file=log.txt --log-level=DEBUG --transfers=1 copy Backup_1:"Drive Backup/Series/GameMovie/" "D:/bruh"

The rclone config contents with secrets removed.

Usual drive config with shared drive. I have my own Client_ID set up.

A log from the command with the -vv flag

2021/04/18 18:57:48 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Sending chunk 1073741824 length 268435456
2021/04/18 18:59:43 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Sending chunk 1342177280 length 268435456
2021/04/18 19:01:16 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Sending chunk 1610612736 length 28746078
2021/04/18 19:01:27 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:01:27 DEBUG : pacer: Rate limited, increasing sleep to 1.037104861s
2021/04/18 19:01:27 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Sending chunk 1610612736 length 28746078
2021/04/18 19:01:37 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:01:37 DEBUG : pacer: Rate limited, increasing sleep to 2.507606713s
2021/04/18 19:01:37 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Sending chunk 1610612736 length 28746078
2021/04/18 19:01:47 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:01:47 DEBUG : pacer: Rate limited, increasing sleep to 4.249160081s
2021/04/18 19:01:47 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Sending chunk 1610612736 length 28746078
2021/04/18 19:01:57 DEBUG : pacer: low level retry 4/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:01:57 DEBUG : pacer: Rate limited, increasing sleep to 8.322179832s
2021/04/18 19:01:57 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Sending chunk 1610612736 length 28746078
2021/04/18 19:02:07 DEBUG : pacer: low level retry 5/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:02:07 DEBUG : pacer: Rate limited, increasing sleep to 16.836235733s
2021/04/18 19:02:07 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Sending chunk 1610612736 length 28746078
2021/04/18 19:02:17 DEBUG : pacer: low level retry 6/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:02:17 DEBUG : pacer: Rate limited, increasing sleep to 16.765919234s
2021/04/18 19:02:24 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Sending chunk 1610612736 length 28746078
2021/04/18 19:02:34 DEBUG : pacer: low level retry 7/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:02:34 DEBUG : pacer: Rate limited, increasing sleep to 16.300931621s
2021/04/18 19:02:41 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Sending chunk 1610612736 length 28746078
2021/04/18 19:02:51 DEBUG : pacer: low level retry 8/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:02:51 DEBUG : pacer: Rate limited, increasing sleep to 16.281528041s
2021/04/18 19:02:57 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Sending chunk 1610612736 length 28746078
2021/04/18 19:03:07 DEBUG : pacer: low level retry 9/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:03:07 DEBUG : pacer: Rate limited, increasing sleep to 16.586505706s
2021/04/18 19:03:14 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Sending chunk 1610612736 length 28746078
2021/04/18 19:03:24 DEBUG : pacer: low level retry 10/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:03:24 DEBUG : pacer: Rate limited, increasing sleep to 16.511431536s
2021/04/18 19:03:24 DEBUG : Confronting.a.Serial.Killer.S01E01.Getting.Away.With.Murders.720p.AMZN.WEB-DL.DDP.5.1.H.264-FLUX.mkv: Received error: googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded - low level retry 1/10
2021/04/18 19:03:31 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/04/18 19:03:31 DEBUG : pacer: Rate limited, increasing sleep to 16.95857248s
2021/04/18 19:03:47 DEBUG : pacer: Reducing sleep to 0s
2021/04/18 19:04:04 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:04:04 DEBUG : pacer: Rate limited, increasing sleep to 1.877664097s
2021/04/18 19:04:05 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:04:05 DEBUG : pacer: Rate limited, increasing sleep to 2.277283065s
2021/04/18 19:04:07 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:04:07 DEBUG : pacer: Rate limited, increasing sleep to 4.818711806s
2021/04/18 19:04:09 DEBUG : pacer: low level retry 4/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2021/04/18 19:04:09 DEBUG : pacer: Rate limited, increasing sleep to 8.257751028s
2021/04/18 19:04:14 DEBUG : pacer: low level retry 5/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:04:14 DEBUG : pacer: Rate limited, increasing sleep to 16.056429471s
2021/04/18 19:04:23 DEBUG : pacer: low level retry 6/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:04:23 DEBUG : pacer: Rate limited, increasing sleep to 16.01388682s
2021/04/18 19:04:38 DEBUG : pacer: low level retry 7/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:04:38 DEBUG : pacer: Rate limited, increasing sleep to 16.287224138s
2

hello,

the rclone command seems to be downloading files, not uploading.
and the top lines of the debug log are missing?

https://developers.google.com/drive/api/v3/handle-errors#403_user_rate_limit_exceeded
the 403 is a message from google, and in response, rclone is behaving correctly.
nothing rclone can do about that. usually takes 24 hours for the quota to reset.
i think this is more common with shared files.

when rclone exits, what is shown at the end of the debug log?

we have gdrive experts and if what i wrote is not correct, they will update this post.

3

sorry I posted the wrong command, just switch the src and dest

Resolve errors  |  Google Drive API  |  Google Developers
the 403 is a message from google, and in response, rclone is behaving correctly.
nothing rclone can do about that. usually takes 24 hours for the quota to reset.
i think this is more common with shared files.

But afaik if the user limit is exceeded rclone would stop uploading files altogether. Is that not the case?
In my case its uploading completely up to 100% but it never finishes.

4

what does git bash have to do with this? if you think that is the problem, then try without it?

how much data have you upload in the past 24 hours, as there is a limit of 750GB

not sure what the means?
does rclone crash, does it just exit, what is at the end of the debug log?

what error code does rclone give at exit?
https://rclone.org/docs/#exit-code

5

yeah apparantley git bash has nothing to do it, I tried with cmd, but same results.
Rclone doesn't crash nor does it exit, it just sits there at 0/s showing 100%, if I let it sit there for a while, it starts re uploading the same file.

how much data have you upload in the past 24 hours, as there is a limit of 750GB

Around 60-70 GB. Although I did do a server side copy between two drives that was around 500GB. But as I understand those two limits are separate

what is at the end of the debug log?

2021/04/18 19:04:38 DEBUG : pacer: low level retry 7/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2021/04/18 19:04:38 DEBUG : pacer: Rate limited, increasing sleep to 16.287224138s
6

Update: I just tried uploading it on a third drive and it worked this time. I guess it is the User Rate Limit Exceeded issue.

7

there have been a lot more posts recently about issues with gdrive quotas, a lot to talk that google is tweaking the values that trigger the limits.

8

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.