Rclone sync gdrive pacer: low level retry 10/10

CMD line:
2018/12/31 09:00:09 DEBUG : rclone: Version "v1.45-050-g1c01d0b8-beta" starting with parameters ["/usr/bin/rclone" "sync" "dropbox:" "gdrive:" "--retries=5" "--transfers=6" "--checkers=6" "--stats" "60s" "--tpslimit" "3" "--drive-chunk-size" "128M" "--log-level" "DEBUG" "--log-file=/home/plex/logs/syncgdrive.log"]

I still get a lot of pacer: low level retry 10/10

2018/12/31 10:20:09 INFO  : 
Transferred:   	   12.898G / 12.898 GBytes, 100%, 2.751 MBytes/s, ETA 0s
Errors:                10 (retrying may help)
Checks:             11362 / 11362, 100%
Transferred:            4 / 4, 100%
Elapsed time:   1h20m0.5s

2018/12/31 09:06:05 ERROR : crypt/rijbp68ecdsepkm033j05hjoos/2ne792n2ein8fc50nh9dghf2eg/6ujr39a7dl6bgvegc8r6sdh6g4: error reading destination directory: couldn't list directory: googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded

@ncw any way to get the actual file names during copy eg

  • crypt/hm2ka8or5qp1pe…4fbfjkjl0t6obos134lo: 64% /8.712G, 7.195M/s, 7m19s
    

I thought could do it with --crypt-show-mapping but its not working.

I think your transfers/checkers are causing the issue. I’d move them down to more like 3/3 and if that works, move up to 4/4 or 5/5 to get no more than 10 total.

I’m not quite sure how the tpslimit plays with those two.

Check out the issue that was opened up and if you’d like to test and provide feedback, that would be great!

You can decode them with rclone cryptdecode

Will test it, well with tpslimit 1 there never should be 403 pacer error, the problem is then its really slow.

I have 150TB+ of data eg 9K+ movies and around 50K TV shows and each have like 2 subtitles that almost 200K.

But I cant use it with sync eg to see decoded names in log file.

This is interesting, much over the 750GB upload limit gdrive enforces
Command line:
/usr/bin/rclone sync dropbox: gdrive: --retries=5 --transfers=10 --checkers=3 --tpslimit=2 --drive-pacer-min-sleep=100ms --bwlimit=35M --drive-chunk-size=256M --log-level=INFO --log-file=$LogFile

2019/01/07 17:04:30 INFO  : 
Transferred:   	    3.745T / 4.292 TBytes, 87%, 33.292 MBytes/s, ETA 4h46m54s
Errors:                11 (retrying may help)
Checks:            140395 / 140395, 100%
Transferred:          165 / 216, 76%
Elapsed time:  32h46m0.4s
Transferring:
 *     crypt/0ibcad98vheuku…1niqjke6df4of3ap9oh0: 22% /50.926G, 3.316M/s, 3h22m42s
 *     crypt/0ibcad98vheuku…1v9bdn1loi4mfdt77ou2: 20% /48.405G, 3.238M/s, 3h21m35s
 *     crypt/0ibcad98vheuku…2dpfmbm5m77f5fmm7psk: 21% /56.085G, 3.497M/s, 3h36m8s
 *     crypt/0ibcad98vheuku…7lqjtei3kqgavdt2hkkg: 23% /69.469G, 3.653M/s, 4h9m23s
 *     crypt/0ibcad98vheuku…bku4sdjjnk61lubuelo8: 31% /47.144G, 3.590M/s, 2h32m26s
 *     crypt/0ibcad98vheuku…fdpc49eorf4ccmm80da8: 19% /52.072G, 3.404M/s, 3h29m27s
 *     crypt/0ibcad98vheuku…hsbem9en3oastv8gfcvo: 20% /49.175G, 3.195M/s, 3h27m44s
 *     crypt/0ibcad98vheuku…op0aap34r59ehtli4sgg: 25% /41.127G, 3.772M/s, 2h19m23s
 *     crypt/0ibcad98vheuku…rmh25clh54bq54316uqg: 19% /53.052G, 3.524M/s, 3h27m1s
 *     crypt/0ibcad98vheuku…tgfh873a192i7hboog08: 32% /48.363G, 3.750M/s, 2h29m11s

Interesting… What do you think is causing that?

Just for grins I tried with the exact parameters you have and was banned at ~750G for the day.

It seems it was copying those big 50GB files over and over without the ban
posted findings and logs here https://github.com/ncw/rclone/issues/2880#issuecomment-452226385

1 Like

This works great now, good speed and no 403 errors & upload limit’s

CMD.
/usr/bin/rclone sync dropbox: gdrive: --retries=5 --transfers=4 --checkers=4 --tpslimit=2 --drive-pacer-min-sleep=100ms --bwlimit=9M --dropbox-chunk-size 150M --drive-chunk-size=512M --log-level=INFO --log-file=$LogFile

last log entry ( still going )

2019/01/22 15:31:12 INFO  :
Transferred:        8.211T / 8.840 TBytes, 93%, 8.968 MBytes/s, ETA 20h26m8s
Errors:                32 (retrying may help)
Checks:            227174 / 227174, 100%
Transferred:         1889 / 2170, 87%
Elapsed time:  266h40m0.6s
Transferring:
 *     crypt/qbu2chuhnvlm4g…a7e27sjkg2gnoqh6rcci: 61% /4.041G, 2.251M/s, 11m40s
 *     crypt/qbu2chuhnvlm4g…ce3u1jfauok15sj6m80e: 98% /3.683G, 2.257M/s, 17s
 *     crypt/qbu2chuhnvlm4g…iecaat1saaik3r32kmdu: 23% /4.094G, 2.257M/s, 23m45s
 *     crypt/qbu2chuhnvlm4g…sq05ko7c53b49uin31fu: 50% /4.127G, 2.233M/s, 15m40s

@ncw will flag --drive-pacer-min-sleep be added beta releases

1 Like

Amazing!

Yes I plan to put this in the release. I have one more tweak here

https://beta.rclone.org/branch/v1.45-059-gf935acb2-fix-2880-drive-pacer-beta/ (uploaded in 15-30 mins)

This lets the pacer burst when it hasn’t used any calls recently - this should be what rclone mount needs.

The binary has a --driver-pacer-burst flag to control this.

2 Likes

Will wait for linux release

I messed up the build, try

https://beta.rclone.org/branch/v1.45-103-gc9e50b32-fix-2880-drive-pacer-beta/ (uploaded in 15-30 mins)