Just realized there’s one more advantage there - checksums, which are still very much a problem with rclone’s crypt. With encfs under rclone syncing with checksums will Just Work™.
So a cheaper method (in terms of not using all your $300 credits). I spun up a 2012 Server with basically no storage, installed ExpanDrive and am doing a copy from ACD to Gdrive. I’m getting around 40-50MB/s. A few things here, you don’t have to rclone to Gdrive later, you’re already doing that with the copy from ACD to GDrive. Downside you can only do 10TB blocks at a time because ExpanDrive presents that both drives are 10TB each. It’s an alternative if you don’t want to worry about linux, not sure in the end which is faster.
multcloud shows also only 10TB each (gdrive and acd)
but I guess this is just the GUI
it can do more
Rclone say me “the system is Not functioning” after a Lot of Transfers from expandrive. Habe you the Same Problem?
How are you guys getting 700MBit/s here is my average download using the 10 thread method.
For me, it depends on the file size. I have some .mov home video’s that are 40gb or so, during these transfers I hit the top speed. On folders with a lot of smaller files the speeds are less. Although all my transfers are now queit fast.
I’m running a n1-highcpu-2 (2 vCPU’s, 1,8 GB RAM) on Google Compute Engine at europe-west1-d
since my ACD is in Germany.
After the night speeds went up:
maybe 10 is too much. maybe the region you provisioned your instance in.
i am running europe-west1-c
Yeah I am US East 1 d
Your upload method to google drive is what? is this also using odrive? and if so what is the proper way to do this just mv the amazon folder to the google folder and sync?
Not sure what the right US location is if your ACD is in the US located as well. I’m now getting lower speeds too though, probably just per file based since it looks like it already fetched the larger files.
I’m not yet uploading to G-Drive, will try rclone first. Not sure what the right way would be with odrive, will let you know how I did it when all my data is downloaded to the GCE instance.
Can someone point me to the guide to using odrive to sync between ACD and gdrive on Linux? I have 30tb I need to get off of ACD.
using rclone sync with google drive remote to upload.
What kind of speeds are you getting with this method?
speeds seems to have picked up now:
So, my transfer finally finished after ~28 hours. I decided to not copy my Arq back-up’s as that was another 5TB. I did a re-run of the command I used before to be sure nothing was missed but it looks like it already downloaded everything. (no .cloud files anymore).
I’m now copying to G-Drive with the following command, I’m starting with the folder called Films
so I can see if everything goes smoothly before I transfer the other stuff:
rclone copy ~/odrive-agent-mount/Amazon\ Cloud\ Drive/Media/Films/ gdrive:Media/Films
I’ve used a custom client_id and client_secret because I would need it later for plexdrive anyway.
Speeds are very nice Avg: 1.27 GBit/s. Should be done tomorrow evening I hope (got some tv show catching up to do than!) After that, it’s time to request a refund from Amazon…
What do you mean by your own client id, you did this with Google drive?
Can someone please help me with the 2TB limit on the VM? I have a business account and I upgraded to a paid account when prompted. For some reason no matter what size I select when setting it up, sda1 maxes out at 2TB when I check via ssh. Am I doing something wrong when creating the VM? Any advice is much appreciated.
You need to add the second drive so use the 10gb for the os and create another disk as 10240GB after you add it to the vm when it boots you need to format and mount the disk. To get over the 10tb you need to request an increase from the quota page
Never mind figured it out.