Problems copying files with rclone

Not sure what the question is as that is a 2 seconds of a log.

2022/10/15 05:00:22 ERROR : Fatal error received - not attempting retries
2022/10/15 05:00:22 INFO  : 
Transferred:       42.023 KiB / 63.598 GiB, 0%, 14.012 KiB/s, ETA 7w6d2h2m1s
Errors:                16 (fatal error encountered)
Checks:               334 / 334, 100%
Transferred:            0 / 265, 0%
Elapsed time:         5.2s

2022/10/15 05:00:22 DEBUG : 25 go routines active
2022/10/15 05:00:22 Failed to copy with 16 errors: last error was: googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded

It says the limit was exceeded, but I can still copy manually, so I don't understand why it gave this error and stopped copying.

Me either since I can only see 5 lines of a log file.

I don't know what command you ran.
I can't see the whole log.

To get help, you have to help us help you and share the full details.

What command did you execute?
What's the full log look like?

Command:
rclone copy $ORIGEM $DESTINO -v --transfers 8 --drive-stop-on-upload-limit -vv --log-file $DIRL/rclone.log

LOG:

The command you shared and what actually executed aren't the same.

2022/10/15 05:21:58 DEBUG : rclone: Version "v1.59.1" starting with parameters ["rclone" "copy" "mirabel-crypt:" "jelly-crypt:/BKP" "-v" "--transfers" "8" "-vv" "--log-file" "/home/jelly/Becape/rclone.log"]

You aren't using drive stop on upload limit so it'll keep going if it hits an upload error. The actual error from the log though is a download quota.

2022/10/15 11:12:15 ERROR : Series/American Horror Story/Season 10/American Horror Story - S10E04 - Blood Buffet - WEBDL-1080p - h264 EAC3 [EN].nfo: Failed to copy: failed to open
source object: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded

Which is

--drive-stop-on-download-limit

Google drive (rclone.org)

I'm using the limit, I just took a moment to see if it made a difference, and it didn't.
This log is quite big, I left it running yesterday with the --drive-stop-on-download-limit parameter active, you only took a part that I had taken to test., It is running with it active now, but the limit is locked, I have to wait.

If you’d like to share a log, I can look at it.

I can only give advice on what I see in the logs.

The flags work when used so if there is an issue, you have to share a log with it.

Here's the log you asked for, as you can see I'm using the parameter to copy, and it says it exceeded the limit, but actually the limit has just been released, I can manually copy files from one drive to the other, but using rclone it is showing an error.

LOG:

I'm looking for this, where do I access information?


--bwlimit 8500 and --max-transfer-total 650GB

rclone copy $ORIGEM $DESTINO -v --bwlimit 8500 --max-transfer-total 650GB --drive-stop-on-upload-limit -vv --log-file $DIRL/rclone.log

It looks like the --max-transfer-total flag no longer exists.

Error: unknown flag: --max-transfer-total
Usage:
  rclone copy source:path dest:path [flags]

Flags:
      --create-empty-src-dirs   Create empty source dirs on destination after copy
  -h, --help                    help for copy

Use "rclone [command] --help" for more information about a command.
Use "rclone help flags" for to see the global flags.
Use "rclone help backends" for a list of supported services.

2022/10/16 01:58:51 Fatal error: unknown flag: --max-transfer-total

Right - you can see in the log at the end, you've hit your upload limit:

2022/10/16 01:39:28 ERROR : Series/American Gods/Season 2/American Gods - S02E01 - House on the Rock - WEBDL-1080p - h264 EAC3[EN+PT] [PT].mkv: Failed to copy: googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded
2022/10/16 01:39:28 ERROR : Cancelling sync due to fatal error: googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded

I have no idea what that means as the log is showing you've hit the upload quota.

What does that mean? You are copying them how? Using the WebGUI? Using what?

That's because you are using a non existent flag. It's called:

      --max-transfer SizeSuffix              Maximum size of data to transfer (default off)

You can find all the flags here:

Global Flags (rclone.org)

As you can see, it's the same error, the limit was exceeded, but it doesn't stop trying to copy, wasn't the flag just to stop copying when this error occurred?

2022/10/17 23:56:05 INFO : Series/Suits/Season 7/metadata/Suits - S07E15 - Tiny Violin - WEBDL-1080p - x264 AC3[PT+EN].jpg: Copied (new)
10/17/2022 23:56:06 DEBUG : Series/Supernatural/Season 2/Supernatural - S02E02 - Everybody Loves a Clown - Bluray-1080p - x264 AAC[EN+FR+PT]-thumb.jpg: md5 = a3e3c7545857838000c4982b668acafe OK
10/17/2022 23:56:06 INFO : Series/Supernatural/Season 2/Supernatural - S02E02 - Everybody Loves a Clown - Bluray-1080p - x264 AAC[EN+FR+PT]-thumb.jpg: Copied (new)
2022/10/17 23:56:06 ERROR : Series/The Big Bang Theory/Season 10/The Big Bang Theory - S10E13 - The Romance Recalibration - WEBDL-720p - h264 EAC3[EN+PT] [PT].nfo: Failed to copy: failed to open source object: open file failed: googleapi: Error 403: The download quota for this file has been exceeded., downloadQuotaExceeded

That is what I posted a few lines up.

This is the command I'm running

rclone copy $ORIGEM $DESTINO -v --bwlimit 8500 --drive-stop-on-upload-limit -vv --log-file $DIRL/rclone.log

Do you see:

In that command?

Wow, I'm sorry, now that I saw it, there's an upload command and a download command :confused:
I'll put download and upload, thanks.

1 Like

Generally, the majority of folks hit the 'upload' quota as that's 750GB per day. The download quota is not documented but most people think 10TB a day per user and a bunch of other restrictions per file or whatnot (not documented so people just make assumptions).

Hopefully that fixes it for you!

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.