Copying Data from Google to LayerOnline or DropBox

Okay I've got the dreaded email, I'm ont the shortlist for a google smack down. So I've decied it's either LayerOnline or Dropbox (if they re open single user accounts)

How are people moving their 50+TB of data, I know my local connect would take months, and annoy the family :laughing:

Any suggestions?

I guess the most common answer would be rent a VPS with a good internet link and decent cpu for a month, a use rclone for the copy.

Rent a Nocix dedicated server: $20, unlimited bandwidth 1gbps

hello and welcome to the forum,

yes, good point

blackhost is popular choice in the forum, that i also have used for years.
1Gbps=$11.99

and might use --bwlimit to stay below the gdrive 10TiB/24Hour

Ordered LayerOnline Yesterday Morning.
Got Drive Details about Lunchtime.
Ordered Blackhost
Got Server Details Early Evening.
Setup Rclone by Late Evening.

Hit the 750Gb limit sometime over night :laughing:
Restarted with a 10M Limit (I figure there will be some slow times naturally so it might be Okay)

Got to wait to upload some more now :slight_smile:

I have noticed that some of the larger files are going to 100% several times, suggesting some sort of fail\retry. But no obvious errors given. Anybody suggest anything?

What is your upload speed (rclone sync or copy) on gdrive remote to dropbox remote?
And which server did you got?

can you post a rclone debug log?

@alneven I didn't use dropbox, I went with LayerOnline until dropbox bring back in the single user accounts again. I'm hitting the 750Gb limit repeatedly, so keep knocking it down slowly.

@asdffdsa I think the retries are when I'm close to the 750Gb so a little time goes by, giving me a little more allowance, and the update succeeds, on the second/third attempt.

Well That didn't last Long, now trying again with dropbox on LayerOnline :laughing:

What sort of speeds should I be uploading to dropbox with, don't want to blow that one up aswell :slight_smile:

tl;dr please provide more details......

what is that?

not sure what you are asking?

https://forum.rclone.org/t/rclone-copy-google-drive-to-dropbox-slow/39091/6

I started with bwlimit 75 and another server with bwlimit 50
But after a day/ few hours I reached some limit, and I had to stop
Then I waited 5-6 hours, started again with only one server and bwlimit 75
It went well for another 1 day, and I had to wait 5-6 hours
After that I let it on 75M, and it was going with only a few errors, like too many requests or some i/o errors
But after that the speed just went down, to 0kb/sec to max 50-60kb/sec
Without bwlimit I reached after restarts over 100MB/sec but only for minutes.

Then I decided to go back to the 2nd server I haven’t used for days, and started the copy process 5 days ago with bwlimit 60M:50M, and it is still running since without any issues

LayerOnline’s google drive got the limitation as well? Or did google closed the account?

@alneven I recieved this from LayerOnline

We just got the 5TB limit message on PhotoKeep's Google admin console, all files will be on read-only status at around end of July. We are offering migration path to our Dropbox offering.

If you are using our Shared Drive Personal plan, you can do an in place upgrade to Dropbox. Or just order a new Dropbox plan and we will cancel your shared drive and refund the difference.

If you are using our Dropbox Bundle, simply stop using Google Shared Drive. We will remove the Google drive in description and from your account.

We will make sure everyone's data is off Google before we shut it down. We will create a ticket for each user to confirm.

We recommend you to use a cheap dedicated server for the data transfer such as nocix. Follow our guide to mount Google and Dropbox using rclone, create service accounts for Google and let it run 24/7.

Rclone Setup with Google Service accounts Using Sa-gen and Sasync - Knowledgebase - LayerOnline

Please let us know if you have any questions or concerns.

Thanks,

Thanks for the speed info

@asdffdsa It looks like they are now offering dropbox as an alternative, which I migrated to even though it is quote a price bump, from the google offering. Still looking for cheap alternatives as a backup, for the backup as I don't have faith that LAyerOnline and Dropbox are going to last either. Or that Dropbox won't realise that there for some of us Unlimited isn't just a marketing term :laughing:

yeah, i agree and all the more so if the files are backups, not just the latest season of some tv show.

and based on this webpage, with all the mistakes and terrible advice, i have no negative confidence in layeronline.
https://client.layeronline.com/knowledgebase/236/Rclone-Setup-with-Google-Service-accounts-Using-Sa-gen-and-Sasync.html

fwiw, this is what i do with backup files.

  1. most recent backups goto wasabi. $6.00/TiB/month
  2. older backups goto aws deep glacier, $1.00/TiB/month

layeronline dropbox business team account requires 2step authentication is that gonna make rclone freak out and demand I 2step all the time?

I am trying to buy from black.host but for some reason in the cart checkout menu none of the configurable options like location of server can be clicked on or altered, it's weird?

Are there just no other location options or is the page slash my browser bugged out?

best to contact them direct.

as a test, just now, i did the following, so far, no problems:
--- log into black.host website
--- ssh over tailscale into the vm, attach to tmux session

Anyone know how long I'll be stuck on pending for my new black.host VPS? (aka anyone set one up recently and remember? is it like an hour a day a week etc?)

Aka do I stare at my computer so I can set up rclone asap? Or do I check back tomorrow?

edit: black host is online