2019/04/30 13:16:46 INFO : Google drive root 'gdrive/test': Waiting for checks to finish
2019/04/30 13:16:46 INFO : Google drive root 'gdrive/test': Waiting for transfers to finish
2019/04/30 13:16:46 ERROR : test123.mkv: Failed to copy: googleapi: Error 404: File not found: 1yqnKvS9-ab
ev2YMOvqTTCyGM0g., notFound
2019/04/30 13:16:46 ERROR : abc123.mkv: Failed to copy: googleapi: Error 404: File not found: 1KK5xY
gw4JXanTTcR4T7CvkCJ44., notFound
2019/04/30 13:16:46 ERROR : Attempt 1/3 failed with 3 errors and: googleapi: Error 404: File not found: 1KK5xYgw4JXanbgt8S
TcR48TvkCJ44., notFound
2019/04/30 13:16:47 INFO : Google drive root 'gdrive/test': Waiting for checks to finish
2019/04/30 13:16:47 INFO : Google drive root 'gdrive/test': Waiting for transfers to finish
2019/04/30 13:16:48 ERROR : test123.mkv: Failed to copy: googleapi: Error 404: File not found: 1yqnKvS9-ab
ev2YMOvqP4liGZCTGM0g., notFound
2019/04/30 13:16:48 ERROR : abc123.mkv: Failed to copy: googleapi: Error 404: File not found: 1KK5xY
gw4Tnbgt8STcTCvkCJ44., notFound
2019/04/30 13:16:48 ERROR : Attempt 2/3 failed with 3 errors and: googleapi: Error 404: File not found: 1KK5xYgw4JXanbgt8S
TcR48327CvkCJ44., notFound
2019/04/30 13:16:49 INFO : Google drive root 'gdrive/test': Waiting for checks to finish
2019/04/30 13:16:49 INFO : Google drive root 'gdrive/test': Waiting for transfers to finish
2019/04/30 13:16:49 ERROR : test123.mkv: Failed to copy: googleapi: Error 404: File not found: 1yqnKvS9-ab
ev2TMOvqP4lTCyGM0g., notFound
2019/04/30 13:16:49 ERROR : abc123.mkv: Failed to copy: googleapi: Error 404: File not found: 1KK5xY
gw5JYanbgt8T8327Cvk4J44., notFound
2019/04/30 13:16:49 ERROR : Attempt 3/3 failed with 3 errors and: googleapi: Error 404: File not found: 1KK5xYgw4JXanbgt8S
TcR48TkCJ44., notFound
2019/04/30 13:16:49 Failed to copy: googleapi: Error 404: File not found: 1TK5xTXanbgt8STcR48347CvkCJ44., notFound
Files were originally created from 1.39. When I updated it to the latest 1.47, it shoots these errors. However when I rollback to 1.39, its fine.
Are my files locked to 1.39 version?
Can you share your rclone.conf with keys removed and such and with rclone 1.47 run something simple like a rclone ls remote: somefile -vv and share the full output.
No, it’s not locked 1.39. 1.39 very ancient.
I still use 1.39 as the speeds are faster, using v2. I think its about the transition to v3, my files are all created in 1.39, so I’d imagine it wont do so well with v3.
Have no idea what you mean as I push gigabit with 1.47 without issue. That being said, there are no 1.39 or 1.47 items that cause files to be ‘locked’ to a version of rclone.
That being said, you won’t find much support for anything wrong with 1.39 from Dec 2017.
Previously I could download 1 single file at 900Mbps maxing out my gigabit connection.
But now, its 150-200Mbps. Why is this so?
Bear in mind its large single files at 50-100gb. Downloading at this speed is super slow.
Any commands I could try fixing it, seems weird didn’t have this issue last month.
Made this thread a long time ago. Seems like you’re not aware of v2 or v3. I’ll wait for ncw.
Here would be an example of me copying on 1.47 from my mount:
and pushing almost gigabit speeds.
Here would be an example my daily uploads even close to gigabit speeds.
[felix@gemini logs]$ tail upload.log
2019/04/30 03:30:29 INFO : TV/NCIS Los Angeles/NCIS.Los.Angeles.S10E21.mkv: Deleted
2019/04/30 03:30:54 INFO : TV/World.of.Dance/World.of.Dance.S03E11.mkv: Copied (new)
2019/04/30 03:30:54 INFO : TV/World.of.Dance/World.of.Dance.S03E11.mkv: Deleted
2019/04/30 03:30:54 INFO :
Transferred: 89.200G / 89.200 GBytes, 100%, 80.639 MBytes/s, ETA 0s
I’ve probably done more testing than most on v2 vs v3 and if you search, you can find just a few posts related to GD speed testing on the topic.
I’m even in the post you linked showing differences between v2 and v3 show in fact the same movie I’ve used above and pushing 100MBs.
Here is an example from 1.37 against my current crypt:
[felix@gemini rclone-v1.37-linux-amd64]$ ./rclone ls gcrypt:Test
5227381829 11.22.63/11.22.63.S01E01.mp4
3542381398 11.22.63/11.22.63.S01E02.mp4
3265469214 11.22.63/11.22.63.S01E03.mp4
3135548554 11.22.63/11.22.63.S01E04.mp4
1887543968 11.22.63/11.22.63.S01E05.mp4
Try uploading in 1.39 and then download using 1.47.
One thing to point out. Using crypt has no issue between 1.47 and 1.39. Its the unencrypted ones.
Here are both:
crypt:
[felix@gemini rclone-v1.37-linux-amd64]$ ./rclone -V
rclone v1.37
[felix@gemini rclone-v1.37-linux-amd64]$ rclone copy /etc/hosts gcrypt:
[felix@gemini rclone-v1.37-linux-amd64]$ cd
[felix@gemini ~]$ rclone copy gcrypt:hosts /data -vv
2019/04/30 10:55:09 DEBUG : rclone: Version "v1.47.0" starting with parameters ["rclone" "copy" "gcrypt:hosts" "/data" "-vv"]
2019/04/30 10:55:09 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2019/04/30 10:55:10 DEBUG : hosts: Couldn't find file - need to transfer
2019/04/30 10:55:10 INFO : hosts: Copied (new)
2019/04/30 10:55:10 INFO :
Transferred: 205 / 205 Bytes, 100%, 142 Bytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 1 / 1, 100%
Elapsed time: 1.4s
2019/04/30 10:55:10 DEBUG : 4 go routines active
and non encrypted:
[felix@gemini rclone-v1.37-linux-amd64]$ rclone copy /etc/hosts GD:
[felix@gemini rclone-v1.37-linux-amd64]$ cd
[felix@gemini ~]$ rclone copy GD:hosts /data/ -vv
2019/04/30 10:56:26 DEBUG : rclone: Version "v1.47.0" starting with parameters ["rclone" "copy" "GD:hosts" "/data/" "-vv"]
2019/04/30 10:56:26 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2019/04/30 10:56:26 DEBUG : hosts: Couldn't find file - need to transfer
2019/04/30 10:56:27 INFO : hosts: Copied (new)
2019/04/30 10:56:27 INFO :
Transferred: 205 / 205 Bytes, 100%, 230 Bytes/s, ETA 0s
Errors: 0
Checks: 0 / 0, -
Transferred: 1 / 1, 100%
Elapsed time: 800ms
2019/04/30 10:56:27 DEBUG : 4 go routines active
2019/04/30 10:56:27 DEBUG : rclone: Version "v1.47.0" finishing with parameters ["rclone" "copy" "GD:hosts" "/data/" "-vv"]
2019/04/30 15:27:02 DEBUG : Using config file from "/home/testserver/.config/rclone/rclone.conf"
2019/04/30 15:27:02 DEBUG : testserver: Loaded invalid token from config file - ignoring
2019/04/30 15:27:02 DEBUG : Keeping previous permissions for config file: -rw-rw-r--
2019/04/30 15:27:02 DEBUG : testserver: Saved new token in config file
2019/04/30 15:27:03 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimit
Exceeded)
2019/04/30 15:27:03 DEBUG : pacer: Rate limited, increasing sleep to 1.604176898s
2019/04/30 15:27:03 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimit
Exceeded)
2019/04/30 15:27:03 DEBUG : pacer: Rate limited, increasing sleep to 2.756644293s
2019/04/30 15:27:05 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimit
Exceeded)
2019/04/30 15:27:05 DEBUG : pacer: Rate limited, increasing sleep to 4.403939044s
2019/04/30 15:27:08 DEBUG : pacer: Reducing sleep to 0s
2019/04/30 15:27:14 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimit
Exceeded)
2019/04/30 15:27:14 DEBUG : pacer: Rate limited, increasing sleep to 1.737109904s
2019/04/30 15:27:14 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimit
Exceeded)
2019/04/30 15:27:14 DEBUG : pacer: Rate limited, increasing sleep to 2.169533559s
2019/04/30 15:27:16 DEBUG : pacer: Reducing sleep to 0s
2019/04/30 15:27:16 DEBUG : testserver2018-2: Loaded invalid token from config file - ignoring
2019/04/30 15:27:16 DEBUG : Keeping previous permissions for config file: -rw-rw-r--
2019/04/30 15:27:16 DEBUG : testserver2018-2: Saved new token in config file
2019/04/30 15:27:21 INFO : Google drive root 'testfolder/t': Waiting for transfers to finish
2019/04/30 15:27:21 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimit
Exceeded)
2019/04/30 15:27:21 DEBUG : pacer: Rate limited, increasing sleep to 1.834385013s
2019/04/30 15:27:21 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimit
Exceeded)
2019/04/30 15:27:21 DEBUG : pacer: Rate limited, increasing sleep to 2.115413038s
2019/04/30 15:27:23 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimit
Exceeded)
2019/04/30 15:27:23 DEBUG : pacer: Rate limited, increasing sleep to 4.320951717s
2019/04/30 15:27:25 DEBUG : pacer: low level retry 4/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimit
Exceeded)
2019/04/30 15:27:25 DEBUG : pacer: Rate limited, increasing sleep to 8.488740204s
2019/04/30 15:27:30 DEBUG : pacer: low level retry 5/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimit
Exceeded)
2019/04/30 15:27:30 DEBUG : pacer: Rate limited, increasing sleep to 16.805784851s
2019/04/30 15:27:38 DEBUG : pacer: Reducing sleep to 0s
2019/04/30 15:27:55 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimit
Exceeded)
2019/04/30 15:27:55 DEBUG : pacer: Rate limited, increasing sleep to 1.397269366s
2019/04/30 15:27:55 DEBUG : pacer: low level retry 2/10 (error googleapi: Error 403: Rate Limit Exceeded, rateLimit
Exceeded)
2019/04/30 15:27:55 DEBUG : pacer: Rate limited, increasing sleep to 2.103842309s
2019/04/30 15:27:56 DEBUG : pacer: Reducing sleep to 0s
2019/04/30 15:27:56 ERROR : test123.mkv: Failed to copy: googleapi: Error
404: File not found: 1qIwu6ztb0zFFim., notFound
2019/04/30 15:27:56 ERROR : Attempt 1/3 failed with 2 errors and: googleapi: Error 404: File not found: 1qIt0zFFim., notFound
404: File not found: 1qIwu6tMv4Qb0zFFim., notFound
2019/04/30 15:28:00 ERROR : Attempt 3/3 failed with 2 errors and: googleapi: Error 404: File not found: 1qIwutv4Qb0zFFim., notFound
2019/04/30 15:28:00 Failed to copy: googleapi: Error 404: File not found: 1qIwu6wv4Qb0zFFim., notFo
Here is the -vv. Anyway, if there’s no solution i’ll just continue using 1.39. Its faster for me anyway, I can max 80MB/s unlike 1.47.
You seem to not be using your own API key as that can be fixed here as those rate limits errors will slow things down.
https://rclone.org/drive/#making-your-own-client-id
You missed including a rclone.conf and cut off the part of the debug log that shows commands and the parameters ran so very hard to troubleshoot without information unfortunately.
If you just want to use the old version, by all means, do that. If you want to figure out what the issue is, post the additional info and we can step through it.
Does making your own api get charged by google? As in the $300 trial or something. I’ve used up mine so I don’t think I can do it without paying.
There is no charge for using your own API.
You can do it on your GSuite. You can do it on personal GDrive. Once you have a key, you can connect that API key to any google drive.
ncw
May 1, 2019, 9:05am
14
Note that we made this parameter to use the v2 API for downloads if you want for this purpose.
--drive-v2-download-min-size SizeSuffix If Object's are greater, use drive v2 API to download. (default off)
I assumed OP knew that as that was the thread he linked in his post
1 Like
system
July 30, 2019, 10:10am
16
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.
