Proxy for Amazon Cloud Drive


#1

Update 2018-07-09

This workaround no longer works as the oauth proxy’s API keys have been revoked. Unless you have your own API keys for Amazon Drive you can’t use it with rclone - sorry :frowning:

If you happen to know anyone who works at Amazon then please ask them to re-instate rclone into the Amazon Drive developer program - thanks!


I have developed an oauth proxy which stops rclone having to have encrypted secrets in its source code. The proxy server stores or logs no information and it runs on the very secure Google appengine platform.

I was hoping to use the new proxy server with a new set of official credentials from Amazon but none have been forthcoming yet.

However a very kind rclone user has run this proxy server with a set of his own credentials and allowed it to be used publicly.

I suggest you use this to get your data off ACD.

NB the following won’t work using rclone authorize you’ll have to use the copying the config file method if setting it up on a remote server. If configuring remotely you must use a new beta in both places. If you don’t it will work for an hour before giving you an error when it refreshes the token.

How to configure the proxy server

Updated to use easier config method recently added to rclone.

Firstly download the latest beta or make sure you are using version 1.37 or above.

Now run rclone config to make a new remote like this

No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n
name> acd
Type of storage to configure.
Choose a number from below, or type in your own value
 1 / Amazon Drive
   \ "amazon cloud drive"
 2 / Amazon S3 (also Dreamhost, Ceph, Minio)
   \ "s3"
 3 / Backblaze B2
   \ "b2"
 4 / Dropbox
   \ "dropbox"
 5 / Encrypt/Decrypt a remote
   \ "crypt"
 6 / FTP Connection
   \ "ftp"
 7 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
 8 / Google Drive
   \ "drive"
 9 / Hubic
   \ "hubic"
10 / Local Disk
   \ "local"
11 / Microsoft OneDrive
   \ "onedrive"
12 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
13 / SSH/SFTP Connection
   \ "sftp"
14 / Yandex Disk
   \ "yandex"
Storage> 1
Amazon Application Client Id - required.
client_id> rclone
Amazon Application Client Secret - required.
client_secret> acd
Auth server URL - leave blank to use Amazon's.
auth_url> https://quixotic-module-170120.appspot.com/auth
Token server url - leave blank to use Amazon's.
token_url> https://quixotic-module-170120.appspot.com/token
Remote config
Make sure your Redirect URL is set to "http://127.0.0.1:53682/" in your custom config.
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine
y) Yes
n) No
y/n> y
If your browser doesn't open automatically go to the following link: http://127.0.0.1:53682/auth
Log in and authorize rclone for access
Waiting for code...
Got code
--------------------
[acd]
client_id = rclone
client_secret = acd
auth_url = https://quixotic-module-170120.appspot.com/auth
token_url = https://quixotic-module-170120.appspot.com/token
token = XXXX
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y

Now test it is working with rclone lsd acd:.


Amazon Drive and rclone - any updates?
Rclone has been banned from Amazon Drive
Unable to register to ACD
Using local ADS credentials?
I Rate ACD @ 0 and they are Rate Limiting my to 0 as well
Amazon Cloud Drive proxies/workaround
ACD oauth proxy broken!
ACD BAN/Alternatives discussion (error HTTP code 429)
ACD connection problem came back - HTTP error code 429 appears again
Proxy for Amazon Cloud Drive - authentication issue
Amazon Cloud Drive support status?
Failed to copy: File names must not contain DELETE
Does rclone ignore file that are currently being copied?
#2

Follwed instructions but get :

We’re sorry!
An error occurred when we tried to process your request. Rest assured, we’re already working on the problem and expect to resolve it shortly.

Error Summary
400 Bad Request
Unknown client_id
Request Details
access_type=offline
client_id=rclone
redirect_uri=http%3A%2F%2F127.0.0.1%3A53682%2F
response_type=code
scope=clouddrive%3Aread_all+clouddrive%3Awrite
state=0f6f0464e08a9331d2d4363e35d47b39


#3

Are you using the latest beta?


#4

I got the same thing too. I authorized on a PC with a web browser and when I ran the rclone authorize command to authorize on my Pi, I got a ‘We’re sorry!’ error. Fortunately, I was just able to copy the token produced on the windows box to get my Pi working. I’m now transferring all my files elsewhere.


#5

You’ll need the latest beta in both places, and also the auth_url and token_url in both places. In fact I’m not sure the remote config will work…


#6

Beta in both places. Auto-config worked and produced a Token visible in my command prompt. Remote config produced an error, but I got around that by stealing the token from my windows box. I have it working now on both machines. It’s just easier to use my Pi since it’s always on and rclone isn’t a very processor intensive task.


#7

Yes, was using latest downloaded betas.

** edit - it was failing trying the remote - headless with beta at both ends.

I ran it as above just on the local machine and worked fine.
Then copied contents on the .rclone.conf file manually back to the headless machine and all is good…


#8

Yes I don’t think the remote config will work with this - I’ll adjust the post.


#9

Beautiful. Thanks to that rclone user!

BTW, acdcli had a feature of their proxy where you could get the source of the currently running version of the server like this (minus the secrets):

https://quixotic-module-170120.appspot.com/src

the ‘/src’ url would dump the source code output. Might be nice for people to verify what they are hitting (not that the output can’t be manipulated, I suppose). Just a thought…

EDIT; I guess this is compiled so that may be a bit more tricky. :slight_smile: But it sure would be nice to know that the proxy hasn’t been tampered with. I guess you have to trust someone at some point! :slight_smile:


#10

remote config works if you use SSH port forward. Just tested

Fantastic work NCW


#12

Is it just at my end it seems like the “refresh” isn’t working e.g. the token only works for about an hour?


#13

The refresh should work I’ve tested it by invalidating the token in the config file and getting it refreshed. Check you got the token_url correct in the config.


#14

waaaaaaaah half-hearted… thats the word.
but thanks for your rescue tries.


#15

Working perfectly.

THANKS, A LOT …

I have found the problem two hours ago and I have returned here after lunch just to found your smart solution.

Trying to download back of the secured stored files now!.

What a pity what they do … I hope they will offer you an alternative solution for those that weren’t abuse of the unlimited policy.


#17

Dear ACD team, just allow us to get our data off your servers and you’ll never hear from us again. Thanks.


#18

I’m getting an error on beta-latest-windows-386 about “AcquireSRWlockExclusive” not found in KERNEL32.dll - using my Windows 2003. Unfortunately I can’t switch to Win2k8 or 2k12, is it possibility to fix it in next release?


#19

Thanks a ton!

Moving swiftly :slight_smile:


#20

Working perfectly. Now let’s hope gsuite doesn’t backtrack on its unlimited promise or enable the single users limit… :sweat:


#21

Worked for a while, however I now keep getting the error below, despite just having made a new token. (Note, using a Amazon UK account)

Failed to copy: Get REMOVED_cdws.eu-west-1.amazonaws com/drive/v1/nodes?filters=parents%3A%22JI3pwh7VSdWaAPjS_fTUjg%22+AND+name%3A%220n424ue6fbsjjfcf3phtlkjp0ekmnnbrn4h8v1lsq0cp6qru2dmh26qitp3nlvp2g3639tqgf84rs%22: oauth2: cannot fetch token: 401 Unauthorized
Response: {“error_description”:“Client authentication failed”,“error”:“invalid_client”}

I’ve tried upgrading to the latest Rclone today, still get the same. I run the auth on a Windows machine, and then copy the rclone config to my linux box overwriting the old .conf. Works for about an hour, and gives the error.

Content of config file is:;

[acdUk]
type = amazon cloud drive
client_id = rclone
client_secret = acd
auth_url = https://quixotic-module-170120.appspot.com/auth
token_url = https://quixotic-module-170120.appspot.com/token
token = REMOVED


#22

Can you make an issue about that on github please? I suspect that is caused by the build of rclone which includes mount for windows. I also suspect a build without the windows mount code will work. If you make an issue then I’ll help you try the alternatives there.

Thanks