Animosity022 gave a bunch of good answers, and while I'd defer to him, I thought I'd chime in and give my own two cents.
It really is just a game of how confident and comfortable you are with it. I know google's pretty good with maintaining user privacy (I know they've done shitty things in the past, what tech company hasn't) but from my own personal experiences and from other people, they don't really care what you're storing so long as it's not shared publicly.
If you're putting the stuff up in an unencrypted folder, and just passing out the link to it like Halloween candy, then I'd be worried.
I'm storing over 5tb in an encrypted mount, and I feel totally safe. A few friends of mine have access to the folder and I've given them the encryption keys (I know, not a very safe practice, but I trust them and they only have access to certain folders within google).
Obviously, if you don't feel comfortable with it, don't do it.
TL;DR: Don't make "noise". Lay low, store your data, and odds are you'll be fine.