Also received google email today
Interesting, thanks!
Eyeing the lower tier SX64 which could work with some slimming. Maybe some good black Friday offer will be on the table.
Do you know if it possible to add space once on a plan?
Have started comparing to local setup, ~80TB would be around $2000 and sufficient for me. But some data would be best to have off site for backup purposes so need to complement a few TBs remote regardless.
Hetzner has already very competetive prices.
When they do Black-Friday-discounts the only thing to expect is a cut of the one-time setup-fee really. It is something yes, but you most like won't get any cuts on monthly prices or so. Their server-auctions are also all without setup-fee btw and they have also advanced high-storage configurations there (f.e 15 x 10TB-Server for ~230$ - could be less i don't know if it shows european prices with VAT to me)
Unlimited Verizon?
1 Like
Looks like a decent alternative to spin up new storage. One issue is that there is the 50GB per day limit which means you can';t really migrate existing data over in a timely fashion.
The usually incredibly annoying sluggishness around the Google Shared Drives appears to provide an opportunity to reset the suspension timer. Just, if more time may be needed to migrate the contents.
+ How to reset the Google Workspace suspension timer
Stage 0: - This operation needs a separate Google Workspace domain
for holding the content temporarily
- A fresh or trial account is fine
Stage 1: - Empty the main drive ("My Drive"), by moving everything to shared drives
- Drag and drop operations to a shared drive has a 100k file limit per op
- This operation is allowed even when the drive is in a read-only state
- The other way is not, it is only allowed when the drive is below limit
- Share the shared drives with the receiving account
Google Drive (Donor account):
+-----------------------------------------------------------------------------------+
| |
| +----------------+ +----------------+ +----------------+ +----------------+ |
| | Folder 1 | | Folder 2 | | Folder 3 | .... | Folder X | |
| | Max 100k files | | Max 100k files | | Max 100k files | | Max 100k files | |
| +-------+--------+ +-------+--------+ +-------+--------+ +-------+--------+ |
| | | | | |
| | +----------------+ | | |
| | | | +----+ |
| | | +--------------------------------+ | |
| | | | | |
+-+-------v-v--v------+---+------------------+------+--------------v---+------------+
+-------------------+ +------------------+ +------------------+
| | | | | |
| Shared Drive 1 | | Shared Drive 2 | .... | Shared Drive X |
| Max 400k files | | Max 400k files | | Max 400k files |
| | | | | |
+-------------------+ +------------------+ +------------------+
Stage 2: - At the receiving account, move everything to the main drive
- This will change the ownership for the files to the new account
- The change ownership operation is slow (~1-2k files/minute), meaning that
the storage usage for the folders moved will not be updated in real time
- Make sure to move all folders before the storage usage rises over quota
- An initial folder move with i.e. 50k zero-byte-files will buy extra time
- Zero-byte file addition is allowed at donor drive (even if read-only)
Google Drive (Receiving account):
+-----------------------------------------------------------------------------------+
| |
| +----------------+ +----------------+ +----------------+ +----------------+ |
| | Folder 1 | | Folder 2 | | Folder 3 | .... | Folder X | |
| | | | | | | | | |
| +-------^--------+ +-------^--------+ +-------^--------+ +-------^--------+ |
| | | | | |
| | +----------------+ | | |
| | | | +----+ |
| | | +--------------------------------+ | |
| | | | | |
+-+-------+-+--+------+---+------------------+------+--------------+---+------------+
+-------------------+ +------------------+ +------------------+
| | | | | |
| Shared Drive 1 | | Shared Drive 2 | .... | Shared Drive X |
| | | | | |
+-------------------+ +------------------+ +------------------+
Stage 3: - The suspension timer for new account will start
- The donor account will fall below quota eventually. With millions of
files, this can take weeks. The files have been moved already,
but the storage calculation takes time to catch up
- With the storage below quota, the donor account suspension timer is reset
- Reverse the whole operation to get the files back in the donor drive
Suspension timer (months). Triggers when the account storage limit is exceeded.
+------------+-----+-----------+-----+-----+-------------+-----------+----------+
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | X |
+------------+-----+-----------+-----+-----+-------------+-----------+----------+
| Pooled | | Pooled | | | Account | Account | Account |
| storage | | storage | | | scheduled | suspended | canceled |
| limit | | grace | | | for | | |
| exceeded | | period | | | suspension | | |
| | | expired | | | | | |
+------------+-----+-----------+-----+-----+-------------+-----------+----------+
An initial folder move with 50k zero-byte files will add a ~30 min delay before the storage
calculation gets to the real files that will usage storage at the receiving drive.
Uploading the zero-byte files to the donor drive takes around an hour per 50k folder.
$ for i in {00..10}
do
mkdir "zero_byte_files_$i"
pushd "zero_byte_files_$i"
touch zero_byte_file_{00001..50000}.txt
popd
done
1 Like
Thank you! Your response is amazing, if somehow we can implement it. For example, we have 500k+ files. Is there some way to determine how long it would take to migrate back and forth? I'm not quite straight on the reset time...does it reset to stage 1 (pooled storage limit exceeded)?
I use a 1000 files/min ratio as a guideline for that.
Meaning, 500k files would equal 500 minutes - an estimated ~10 hours for the files to settle down at the reception account and be ready to be moved back again.
In the same time frame, the usage at the donation account will be reduced to 0 bytes.
And yes, the suspension timer resets to stage 1 - once again there is 6 months until the next suspension date.
2 Likes
Ok thank you! I will try to figure this out..your instructions are good..but i am a newb at migrations (no experience with CLI). I will go over it again several times, to see if it sinks in.
In your responses, I realized I got the same Google email as the other person (Nov 10), stating the account would be deleted by Dec 9, 2023.
The zero byte file creation is throwing me for a loop. It appears I need to create 50k zero byte .txt files and the snippet you wrote will do it. Is it supposed to be done in Windows > CMD?
Actually, the snippet for the file creation snippet is a sh snippet, meant for the Linux command line.
If you are on Windows, a simple for loop one-liner in CMD will do just as fine:
c:\migration\zero_byte_files_01> for /l %a in (1 1 50000) do type nul > "zero_byte_file_%a.txt"
The zero-byte files are not needed as such, they just help reduce the stress level a little bit 
1 Like
Awesome, thank you! I will try that!
After adding "migration" folder in C:, then typing in CMD prompt, this is the result:
C:>c:\migration\zero_byte_files_01> for /l %a in (1 1 50000) do type nul > "zero_byte_file_%a.txt"
'c:\migration\zero_byte_files_01' is not recognized as an internal or external command, operable program or batch file.
Did I do something wrong? Am I missing something?
Ah, sorry for the confusion - the C:\migration\zero_byte_files_01> part is just my prompt displaying my current working directory, simply leave it out. In your case the prompt is "C:\>", because your current working directory is the root of the C drive.
The for loop will create the files in the current working directory, so maybe create a folder for them first.
C:\> mkdir zero_byte_files_01
C:\> cd zero_byte_files_01
C:\zero_byte_files_01> for /l %a in (1 1 50000) do type nul > "zero_byte_file_%a.txt"
1 Like
Awesome thanks! Worked perfectly. 
I just got an email from Google Workspace a few days ago saying I've got 30 days notice to download 1,100 TB before they will delete my data! A few months ago their support said I would be able to access my data FOREVER (but can't upload anything new) as long as I continued paying my £15 per month. I have a 5.5 MB/s broadband download speed. It would take me 7 years to download all that, not 30 days!
I don't have time to read this massive thread. Can anyone let me know if there's an alternative to Google Workspace with unlimited storage or lots of storage that will let me transfer my data over from Workspace to it via Rclone or some other method? I know a lot of the early suggestions are no longer an option as they ended their unlimited storage.
MultCloud can be used to transfer from Google Workspace to Backblaze which has unlimited storage. Is it any good with fast transfer speeds to Workspace?
Even utilizing full speed of a gigabit-connection you cannot transfer 1100 TB in 30 days. You roughly need 2 hours for 1TB. For 1100TB you need 2200 hours, which equals to ~90 days.
I really don't know if google offers to pull stuff with 10gbit from their servers, i never had the pleasure to own something with a 10gbit-con, but if you really want to save this data and google doesn't extend this time this going to be very expensive.
Sorry but its kinda funny to read that all the datahorders going in panic. Karma for ruin it for the rest of us.
1100TB yea you prolly need all the data ...
Go enterprice and pay up for the service you want.
2 Likes
Then there is Googles limits 10TB/day download, so no connection can pull more than 10tb/day (about 920mb/s) average.
2 Likes
edit: I realized, I just dont care enough anymore to tell off all the idiots.
Baffling. Dropbox was almost shut down for a month due to the influx on their infrastructure. They couldn't expand due to the rampant abuse.
I think you missed the part on the contract that says subject to change.
It's really math as storing x amount for y per month and you were clearly getting a huge steal and you know that. While the run was good, the unlimited era had to come to an end.
With anything that has unlimited, there are always going to be outliers using way more than everyone else which eventually either catches up to them, the service or anyone else.