Syncing crypt FOLDER from Google drive to Amazon is failing

Hello,
i finally uploaded all my content to a crypt folder in google drive.
Next step will be syncing this to my Amazon Cloud Drive and what i have done is:

rclone --verbose --transfers=20 --drive-chunk-size=32M --no-update-modtime --ignore-existing sync secretgoogle:PATH secretamazon:PATH

But what i get is tons of errors:

pacer: low level retry 1/1 (error HTTP code 408: “408 REQUEST_TIMEOUT”: no response body)
Object not found - waiting (10/20)
Giving up waiting for object - returning original error: HTTP code 408: “408 REQUEST_TIMEOUT”: no response body (“408 REQUEST_TIMEOUT”)

I’m doing something bad? What’s the best strategy to keep in sync crypt folder between google and amazon.
What i would like to avoid is uploading to 2 providers in parallel, i prefer using a VPS to do this job.

your help is more than welcome…

Are you sure google did not lock your account for 24h ?
Trying copying one file from it to local drive using rclone copy

I think it’s not the case… as the files are copying but all of them seem to be not found (“Object not found”).
I have upgraded clone to latest beta and it is also giving me the errors and new “429 Too Many Requests”…

The weird thing is that if i go to the target, most of the files are there.

So copying files work you tested it as with google lock you can list and see all files just download is not possible. As for to many requests its “normal” for amazon drive but retries will solve it.

I have checked and i’m not banned, i can download files from Google.
It seems it is copying correctly the file but when checking in Amazon it says Object not found…
something related to checking file size??

Error detected after finished upload - waiting to see if object was uploaded correctly: HTTP code 408: “408 REQUEST_TIMEOUT”: no response body (“408 REQUEST_TIMEOUT”)
2017/01/27 11:08:59 file.mkv: Object not found - waiting (1/416)

When the countdown finish i get this:

file.wmv: Giving up waiting for object - returning original error: Post https://content-na.drive.amazonaws.com/cdproxy/nodes?suppress=deduplication: EOF (“HTTP status UNKNOWN”)
2017/01/27 11:15:35 pacer: Rate limited, sleeping for 598.030433ms (1 consecutive low level retries)
2017/01/27 11:15:35 pacer: low level retry 1/1 (error Post https://content-na.drive.amazonaws.com/cdproxy/nodes?suppress=deduplication: EOF)
2017/01/27 11:15:35 file.wmv: Received error: Post https://content-na.drive.amazonaws.com/cdproxy/nodes?suppress=deduplication: EOF - low level retry 1/10

If you are doing this, make sure you use the latest beta as there is a bug in this kind of transfer: https://github.com/ncw/rclone/issues/902

Also why not sync the encrypted remotes to save decrypting and re-encrypting everything?

As for your query - ACD seems to go through periods of being really unreliable - just re-run rclone until it is all synced.

[quote=“ncw, post:7, topic:888”]
If you are doing this, make sure you use the latest beta as there is a bug in this kind of transfer: https://github.com/ncw/rclone/issues/902 [/quote]
I’m using latest beta

[quote=“ncw, post:7, topic:888”]
Also why not sync the encrypted remotes to save decrypting and re-encrypting everything?[/quote]

[Edited] Understood, but this could be OK for the FULL sync, but the difficulty comes to identify individual folders, i have one big folder with several encrypted folders below it, so if i want to make partial syncs i can not identify which folders to select.

Finally it retries an find correctly most of the objects, and if not, i re-run the rclone command and it is finishing the transfer, so not an issue.
I’m aware of ACD lack of reliability, and also waiting for improvements in rclone mount (affected by google ban), to use alternatively ACD and Google.

Thanks for your amazing work and collaboration with this tool.

This should help with that.

https://github.com/ncw/rclone/issues/1004

It definitely helps, i’m gonna try in a minute…
[Edited] I’ve run a lot… it’s not implemented yet. :slight_smile:

So will --crypt-show-mapping apply to sync, ls, lsd??
This could help a lot to create scripts to do partial syncs without decrypting and re encrypting…