KanKan
(Malcolm)
November 30, 2018, 5:08pm
1
This Week is my first attempt at using rclone or CLI/SSH. I am a windows users and have never had much need for it. Just want to make sure that imam getting the most out of my setup. I have Comcast 1gb internet. Are the speeds below average? I keep reading about people hitting the gDrive daily limit. With my current speeds I will never come close. Is there anything I need to do or can do to improve it?
Transferred: 258.686G / 4.860 TBytes, 5%, 4.500 MBytes/s, ETA 298h12m43s
Errors: 2 (retrying may help)
Checks: 1168 / 1168, 100%
Transferred: 311 / 10335, 3%
Elapsed time: 16h21m0.1s
Transferring:
XXXXXXXXXXXXXXXXXXXX HDTV-720p.mkv: 83% /2.286G, 609.855k/s, 10m56s
XXXXXXXXXXXXXXXXXXXX HDTV-720p.mkv: 98% /2.060G, 599.028k/s, 37s
XXXXXXXXXXXXXXXXXXXX HDTV-720p.mkv: 84% /1.911G, 588.920k/s, 8m46s
XXXXXXXXXXXXXXXXXXXX HDTV-720p.mkv: 5% /988.187M, 605.500k/s, 26m12s
XXXXXXXXXXXXXXXXXXXX HDTV-720p.mkv: 37% /1.844G, 594.347k/s, 33m57s
XXXXXXXXXXXXXXXXXXXX HDTV-720p.mkv: 58% /1.663G, 593.502k/s, 20m21s
XXXXXXXXXXXXXXXXXXXX HDTV-720p.mkv: 30% /2.555G, 633.184k/s, 48m55s
XXXXXXXXXXXXXXXXXXXX HDTV-720p.mkv: 57% /2.118G, 611.313k/s, 25m57s
Are you using your own API Key? Do you have any other logs? I’m guessing you are transferring too many things at once and getting rate limited.
What’s the actual command you are running?
What version of rclone are you using?
KanKan
(Malcolm)
November 30, 2018, 5:42pm
3
Not sure how to get logs still trying to wrap my head around all this.
Haven’t entered in the API info from the Developer Console anywhere. I have the Key just not sure how or where it goes.
The mount command I used is
/usr/bin/rclone mount -v Gdrive: /volume1/Mounts/GoogleDrive --allow-other --dir-cache-time 672h --vfs-cache-max-age 675h --vfs-read-chunk-size 64M --vfs-read-chunk-size-limit 1G --buffer-size 32M
The command I am using for the transfer is
rclone copy -v --transfers=4 --checkers=8 -P /volume1/TV\ Shows/ Gdrive:TV\ Shows\ Backup/
The version is
rclone v1.44
os/arch: linux/amd64
go version: go1.11.1
KanKan
(Malcolm)
December 3, 2018, 8:57am
4
@Animosity022
I am no using my own API key and enabled logging. I am receiving Error 403: Rate Limit Exceeded. I currently have 10 transfers and 2 checkers. I have seen examples where people are running a lot more of each than what I have and aren’t reporting this.
2018/12/03 01:43:01 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/12/03 01:43:01 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/12/03 01:43:01 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/12/03 01:43:01 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/12/03 01:43:03 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/12/03 01:43:03 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/12/03 01:43:05 DEBUG : pacer: low level retry 4/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/12/03 01:43:05 DEBUG : pacer: low level retry 4/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/12/03 01:43:20 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/12/03 01:43:20 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/12/03 01:43:20 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/12/03 01:43:20 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
2018/12/03 01:43:21 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded)
If you are not using your own API key, you are going to see issues like that. Those errors mean you have things configured to do too much as you are getting rate limited.
There isn’t much to do other than slow down the transfers or try to get your own API key.
That’s documented here:
https://rclone.org/drive/#making-your-own-client-id
KanKan
(Malcolm)
December 4, 2018, 6:12am
6
sorry that was supposed to say i am now using my own API key. I am still receiving the Errors with only two transfers and 4 checkers.
Maybe try lower if you are still seeing them?
I run with:
# Move older local files to the cloud
/usr/bin/rclone move /data/local/ gcrypt: --checkers 3 --log-file /home/felix/logs/upload.log -v --tpslimit 3 --transfers 3 --drive-chunk-size 32M --exclude-from /home/felix/scripts/excludes --delete-empty-src-dirs
for my GD.
KanKan
(Malcolm)
December 10, 2018, 6:49pm
8
ok still running into issues. It will work great for about for about 10-20 mins and then I get this over and over.
This is the command I am running.
rclone --exclude “#recycle /" --exclude "@eaDir / ” --exclude “@eaDir /” copy -vv --checkers=3 --tpslimit 3 --transfers=3 --drive-chunk-size 32M -P /volume1/TV\ Shows/ Gdrive:TV\ Shows\ Backup/ --log-file /volume1/homes/Malcolm/rclonelogs/rclonefull5.log
2018/12/10 11:38:16 ERROR : Bob’s Burgers/Bob’s Burgers - S03E01 - Ear-sy Rider WEBDL-720p-thumb.jpg: Failed to copy: couldn’t list directory: Get https://www.googleapis.com/drive/v3/files?alt=json&fields=files(id%2Cname%2Csize%2Cmd5Checksum%2Ctrashed%2CmodifiedTime%2CcreatedTime%2CmimeType%2Cparents%2CwebViewLink)%2CnextPageToken&pageSize=1000&prettyPrint=false&q=trashed%3Dfalse+and+(‘1csc1v3_jDmL3PMzoj9wNabK5s5MUNcOd’+in+parents)+and+(name%3D’Bob\‘s+Burgers+-+S03E01+-+Ear-sy+Rider+WEBDL-720p-thumb.jpg’)+and+mimeType!%3D’application%2Fvnd.google-apps.folder’: oauth2: cannot fetch token: 401 Unauthorized
Response: {
“error”: “unauthorized_client”,
“error_description”: “Unauthorized”
}
That looks like you have a bad API key or something configured.
You can run rclone config and update it.