Can I use Rclone to read "hidden" folders in Google Drive?

It's actually simpler than that with ARQ, just point to the not App folder for backups.

Meh, not really. If anything, it's more a policy thing on Google's part. Google allows developers to hide some data from the normal Google Drive GUI. This is probably a good thing on the whole, since it prevents users from mucking about with configuration files that they don't need to be changing.

In Arq's case, it's completely optional. Arq allows you to choose whether or not to use this "hidden app folder" by checking or unchecking a box. Exactly the same data is stored either way. Exactly the same data can be deleted either way. It's still "your" data. You're not accessing any of the developer's own data. In fact, Arq themselves discourage you from using the hidden app data folder because you can't modify it outside of Arq if you ever needed to.

The single reason to use the hidden app data folder? To prevent the sync clients from getting bogged down with all your Arq backups, since there currently isn't a "selective sync" feature.

So no, this would not reduce the security of either Arq or Google. I suppose being able to access any hidden app data folder, any time, would be against Google/developer policy, but let's not confuse this with an actual, legit security issue.

It doesn't work that way. Even if the developers sent me their client ID in plain text it wouldn't help me. The secret has to be generated using my specific Google account, which they have no access to. They would have to write a separate utility to generate these credentials and pass them onto Rclone, and I suspect they have better things to do--they'd probably request that I not use the hidden app folder.

As mentioned this is exactly what I'm trying to avoid doing.

If you note, that reply was for @asdffdsa as I'm aware of your ask.

Oops sorry about that.

Okay, so it appears there is no real solution to this question; is that correct?

Or if there is, it's not going to be something anyone will write up since some see it as a security issue...I personally don't, you own your own data so I see this more like getting around a policy/design consideration, but meh.

My options are:

  • Keep backing up to the hidden app folder, meaning I'll never be able to find a way to manually access that data if I wanted to, or...
  • Back up to the "visible" part of my Google Drive, meaning clients will constantly see/redownload that data, since there's no "selective sync", or...
  • Backup to a separate Google account

Is that correct?

1 Like

My Google Drive behaves differently.

What do you mean "behaves differently?"

You've found a "selective sync" option in the Google Drive for Desktop app? Pretty sure it's been removed.

My level of constructiveness is continuously adapted to the accumulated level of recognition and appreciation I see from you.

hi, not an expert but
as per this, it has been removed.

and as per this, it has not been removed but there is an *

I'm not sure what you mean? I'm not trying to be purposefully difficult or obtuse. I appreciate that you, and others, are contributing on a volunteer basis. You don't have to provide more info, or any info for that matter, if you're not comfortable.

That said, you posted that your Google Drive behaves differently. Could you share what you mean by this? Do you mean that shared folders work differently than how I've described they work in my account? Or do you mean that you see a selective sync option in the Google Drive for Desktop client for your account? What exactly behaves differently?

Yes, thank you. In my experience, one cannot use selective sync anymore or at least it is not available for my account. This means I'm stuck. If I use the hidden app folder, my data is inaccessible outside of the Arq app. If I don't use the hidden app folder, my data is constantly backed up, then re-downloaded to my machine since I mirror all files to a local folder in the Google Drive app.

I thought this was something I could work around with Rclone, and it seems like others have found the way to do this, but it also seems like it's more complicated than my limited knowledge would permit. The solution may well be to not back up to this Google account in Arq, but I have unlimited storage in that account, so it's really ideal.

OK, but then please try to be more listening and open to accept advice that may not fit in you preferred view or solution. I may know or have (fore)seen things that you didn't, so be prepared to accept answers/advice without resistance or complaining. Though some occasional curiosity/scepticism is fine - I make mistakes too.

My Goggle Drive client is setup using streaming/on-demand downloads - so yes you will see the backup, but it will not redownload to your clients unless you access it.

That makes sense and honestly, is probably Google's preferredway of setting things up.

Iā€™m a bit too stuck in the past I guess but I like using the mirrored option and storing everything offline. In that way I have two copies of my data at any one time.

that is what i do and most anybody serious about backups.

  1. run the backup program, in my case, veeam, to backup to a local backup server.
    you can backup to local storage, usb drive, etc...
  2. rclone copy the backup files to cloud.

The Google Drive client allows you to mark files or folders for offline usage, so this is possible and easily setup if your files a organized in a few top folders (E.g. Documents, Pictures, ...).

You can read more about it here:
https://support.google.com/drive/answer/7329379#Keepoffline
https://support.google.com/drive/answer/7638428?hl=en

I am aware of that but I really don't want to go that route. Those folders are only a cache, so if I Heaven forbid lost access to that Google account, or couldn't open the Google Drive app, I would lose access to them. They also don't support features like APFS snapshots and local Time Machine backups. I just want to mirror files, that's my workflow.

Note: I swear I'm not trying to be argumentative, just explaining my workflow.

I should also mention I have about 2 TB uploaded to the "hidden" Arq folder already so that would be a real pain to re-upload.

It seems I can't readily do what I want. I have seen that others have gotten this to work through something like Charles Proxy, but no one on those threads is sharing how.

Here is how someone accessed hidden data for an Android app. I am guessing it would be simpler for a desktop app? How can I see hidden app data in Google Drive? - Stack Overflow

and here it is my turn to answer:

Good luck!

I appreciate you trying to help the best you can!