Rclone copy server side

#1

i am trying to copy folders in server side as it should be faster … but it works from my dedi, but not on contabo server, i mean, it takes ages to finish and it is not bandwith problem, what can be the problem? google ban? the rclone.conf is the same in all machines (two vps in contabo and my dedi) route to google from contabo? the rclone copy from local works ok in three but server side … takes long and after that i think it starts doing no server side but doesnt finish (0% in all files)

#2

Server side copies are on the server so doesn’t matter where anything is located.

The quota daily for server side copies is 100GB.

Do you have any logs? Command you are running? rclone version is what? Did you make your own Client ID/API key? What is ages to finish / 10 minutes? 20 minutes? 60 minutes? days?

Without any information, it could be anything.

#3

it is with client id/api key and in my machine it takes 18s and much more in the contabo vps. IT IS THE SAME CFG in three machines.

because it is in the server is what i want to do as this

i think server side copy for drive is 750gbs too …

#4

Server side copy is 100GB per day.
750GB is the quota for regular copies.

So you made your own client ID / API key?

If you want to include the command you are using and a log file of the good and bad examples, that would be superb.

#5

WELL

rclone copy teamdrive4:descargado/Dragon\ Ball\ Super\ (2015)\ T4\ [BDrip\ 1080p\ h264\ Dual\ AAC\ Sub]/ teamdrive4:TV\ Super/Dragon\ Ball\ Super/Season\ 04/ -vv
2019/05/07 14:41:56 DEBUG : rclone: Version “v1.47.0” starting with parameters [“rclone” “copy” “teamdrive4:descargado/Dragon Ball Super (2015) T4 [BDrip 1080p h264 Dual AAC Sub]/” “teamdrive4:TV Super/Dragon Ball Super/Season 04/” “-vv”]
2019/05/07 14:41:56 DEBUG : Using config file from “/home/ubuntu/.config/rclone/rclone.conf”
2019/05/07 14:41:59 INFO : Google drive root ‘TV Super/Dragon Ball Super/Season 04’: Waiting for checks to finish
2019/05/07 14:41:59 INFO : Google drive root ‘TV Super/Dragon Ball Super/Season 04’: Waiting for transfers to finish
2019/05/07 14:42:02 INFO : Dragon Ball Super 4x01 .mkv: Copied (server side copy)
2019/05/07 14:42:02 INFO : Dragon Ball Super 4x04 .mkv: Copied (server side copy)
2019/05/07 14:42:02 INFO : Dragon Ball Super 4x02 .mkv: Copied (server side copy)
2019/05/07 14:42:02 INFO : Dragon Ball Super 4x03 .mkv: Copied (server side copy)
2019/05/07 14:42:03 INFO : Dragon Ball Super 4x05 .mkv: Copied (server side copy)
2019/05/07 14:42:04 INFO : Dragon Ball Super 4x08 .mkv: Copied (server side copy)
2019/05/07 14:42:04 INFO : Dragon Ball Super 4x06 .mkv: Copied (server side copy)
2019/05/07 14:42:04 INFO : Dragon Ball Super 4x07 .mkv: Copied (server side copy)
2019/05/07 14:42:05 INFO : Dragon Ball Super 4x09 .mkv: Copied (server side copy)
2019/05/07 14:42:05 INFO : Dragon Ball Super 4x10 .mkv: Copied (server side copy)
2019/05/07 14:42:06 INFO : Dragon Ball Super 4x12 .mkv: Copied (server side copy)
2019/05/07 14:42:06 INFO : Dragon Ball Super 4x11 .mkv: Copied (server side copy)
2019/05/07 14:42:06 INFO : Dragon Ball Super 4x13 .mkv: Copied (server side copy)
2019/05/07 14:42:07 INFO : Dragon Ball Super 4x14 .mkv: Copied (server side copy)
2019/05/07 14:42:07 INFO : Dragon Ball Super 4x16 .mkv: Copied (server side copy)
2019/05/07 14:42:07 INFO : Dragon Ball Super 4x15 .mkv: Copied (server side copy)
2019/05/07 14:42:08 INFO : Dragon Ball Super 4x17 .mkv: Copied (server side copy)
2019/05/07 14:42:08 INFO : Dragon Ball Super 4x18 .mkv: Copied (server side copy)
2019/05/07 14:42:09 INFO : Dragon Ball Super 4x19 .mkv: Copied (server side copy)
2019/05/07 14:42:09 INFO : Dragon Ball Super 4x20 .mkv: Copied (server side copy)
2019/05/07 14:42:09 INFO : Dragon Ball Super 4x21 .mkv: Copied (server side copy)
2019/05/07 14:42:10 INFO : Dragon Ball Super 4x22 .mkv: Copied (server side copy)
2019/05/07 14:42:10 INFO : Dragon Ball Super 4x23 .mkv: Copied (server side copy)
2019/05/07 14:42:11 INFO : Dragon Ball Super 4x24 .mkv: Copied (server side copy)
2019/05/07 14:42:11 INFO : Dragon Ball Super 4x25 .mkv: Copied (server side copy)
2019/05/07 14:42:11 INFO : Dragon Ball Super 4x26 .mkv: Copied (server side copy)
2019/05/07 14:42:11 INFO : Dragon Ball Super 4x27 .mkv: Copied (server side copy)
2019/05/07 14:42:12 INFO : Dragon Ball Super 4x28 .mkv: Copied (server side copy)
2019/05/07 14:42:13 INFO : Dragon Ball Super 4x29 .mkv: Copied (server side copy)
2019/05/07 14:42:13 INFO : Dragon Ball Super 4x30 .mkv: Copied (server side copy)
2019/05/07 14:42:13 INFO :
Transferred: 35.409G / 35.409 GBytes, 100%, 2.009 GBytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 30 / 30, 100%
Elapsed time: 17.6s

2019/05/07 14:42:13 DEBUG : 13 go routines active
2019/05/07 14:42:13 DEBUG : rclone: Version “v1.47.0” finishing with parameters [“rclone” “copy” “teamdrive4:descargado/Dragon Ball Super (2015) T4 [BDrip 1080p h264 Dual AAC Sub]/” “teamdrive4:TV Super/Dragon Ball Super/Season 04/” “-vv”]

BAD
rclone copy teamdrive4:descargado/Dragon\ Ball\ Super\ (2015)\ T4\ [BDrip\ 1080p\ h264\ Dual\ AAC\ Sub]/ teamdrive4:TV\ Super/Dragon\ Ball\ Super/Season\ XX/ -vv
2019/05/07 14:52:44 DEBUG : rclone: Version “v1.47.0” starting with parameters [“rclone” “copy” “teamdrive4:descargado/Dragon Ball Super (2015) T4 [BDrip 1080p h264 Dual AAC Sub]/” “teamdrive4:TV Super/Dragon Ball Super/Season XX/” “-vv”]
2019/05/07 14:52:44 DEBUG : Using config file from “/root/.config/rclone/rclone.conf”
2019/05/07 14:52:48 INFO : Google drive root ‘TV Super/Dragon Ball Super/Season XX’: Waiting for checks to finish
2019/05/07 14:52:48 INFO : Google drive root ‘TV Super/Dragon Ball Super/Season XX’: Waiting for transfers to finish
2019/05/07 14:53:48 INFO :
Transferred: 0 / 29.771 GBytes, 0%, 0 Bytes/s, ETA -
Errors: 0
Checks: 0 / 0, -
Transferred: 0 / 30, 0%
Elapsed time: 1m3.1s
Transferring:

  • Dragon Ball Super 4x01.mkv: transferring
  • Dragon Ball Super 4x02.mkv: transferring
  • Dragon Ball Super 4x03.mkv: transferring
  • Dragon Ball Super 4x04.mkv: transferring

2019/05/07 14:54:19 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 500: Internal Error, internalError)
2019/05/07 14:54:19 DEBUG : pacer: Rate limited, increasing sleep to 1.498717571s
2019/05/07 14:54:19 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 503: Backend Error, backendError)
2019/05/07 14:54:19 DEBUG : pacer: Rate limited, increasing sleep to 2.272121074s
2019/05/07 14:54:19 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 500: Internal Error, internalError)
2019/05/07 14:54:19 DEBUG : pacer: Rate limited, increasing sleep to 4.955776373s
2019/05/07 14:54:19 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 500: Internal Error, internalError)
2019/05/07 14:54:19 DEBUG : pacer: Rate limited, increasing sleep to 8.453238034s
2019/05/07 14:54:48 INFO :
Transferred: 0 / 29.771 GBytes, 0%, 0 Bytes/s, ETA -
Errors: 0
Checks: 0 / 0, -
Transferred: 0 / 30, 0%
Elapsed time: 2m3.1s
Transferring:

  • Dragon Ball Super 4x01.mkv: transferring
  • Dragon Ball Super 4x02.mkv: transferring
  • Dragon Ball Super 4x03.mkv: transferring
  • Dragon Ball Super 4x04.mkv: transferring
#6

What’s the rclone.conf entry for each server with keys and passwords removed?

#7

the same (same files copied)

[teamdrive4]
type = drive
scope = drive
client_id = .apps.googleusercontent.com
client_secret = t4TDAvb1sXjjaPq
service_account_file = /home/ubuntu/tres.json
team_drive = Uk9PVA

#8

This seems pretty similar to:

I’d probably add some feedback there as it seems to be something goofy google is doing.

#9

google or the vps? i dont think it should be google, some communications issue … some fw … but i dont know from whom

#10

It’s all server side so more of a google item. I doubt you can do anything to fix it from your side.

You can “–disable copy” and it won’t move server and that would be a work around.

#11

yes … it is just because i dont know why … i know the workarounds :smiley: thank you for your fast answer

#12

now i can … so it seems it was a failed backend server … what a pity dont knowing how to change it

#13

You’d have to follow the issue above to see when it’s fixed or disable copy.

Those would be your options.

#14

supposedly if you ‘share’ the folder to your destination it will work server-side up to the 100/gb a day quota. I’ve not tried it so YMMV.