As a support resource for any of very large number of researchers on the UC Berkeley campus who seek out the Research IT unit at UC Berkeley, we would like to recommend rclone as a tool to transfer large numbers of files (thousands or more at a time) and/or quantities of data (many GB or multiple TB) to Google Drive as a research data safekeeping solution. This is a common use case at our university. Our campus has negotiated unlimited storage on GDrive for users who authenticate as students, faculty, or staff.
It’s problematic for a significant fraction of our researchers that the default OAuth permission rclone requests is …drive (“full permissive … access to all of a user’s files”). Because our researchers are often also instructors, their Google Drive files may include student data (to which access is restricted by federal regulation in the U.S.), pre-publication manuscripts or articles, and/or other intellectual property of which the researchers are protective. It would be preferable in these cases to grant OAuth permission at the scope …drive.file (“access to files created or opened by the app”), so that the researcher could create a backup or similar folder(s) for use with rclone and thus isolate other data s/he stores on GDrive from rclone access.
This is easy to effect by modifying the file drive/drive.go and recompiling, but it’s a rare researcher who is willing / capable of stepping through the compilation process even if given written, step-by-step instructions. Because the support group I represent is small relative to the number of researchers we’d like to make aware of rclone, it’s not practical for us to sit down with each researcher and recompile for their environment – and certainly not to do so each time a new release of rclone becomes available.
FEATURE REQUEST
Therefore, we’d like to make a FEATURE REQUEST: during the configuration process for GDrive (and, should interest support the dev effort, other platforms for which rclone can be configured), can a feature be added such that the user is prompted to choose a different scope than the default if s/he wishes?
I’m new to this community, and would appreciate direction if it would be more/equally helpful to weigh in on GitHub’s issue #337, perhaps more briefly with a link to this forum post for the context/use case info.
One thing to bear in mind about that is that though rclone requests such permissions, the tokens are stored only on the users computer and nowhere else. They do not go though any rclone servers at any time - they are just between the user and google. So I don’t think it should be a problem in practice.
That said, I’m sympathetic to your use case
Here is a similar issue #1799 which is about requesting drive.appdata
drive.appdata gives rclone access to a private area accessible only by the rclone application. It isn’t visible through the drive web interface
I’m not 100% clear on the difference between that and the drive permission. Presumably rclone can create files but what does it mean by acccess must be granted to other files - how does that work?
Absolutely! If you want to us to go ahead with this, then please make a new issue on github. I’ll probably try to roll up all the drive scope issues at the same time so the user could choose…
A selection of one of
drive
drive.readonly
drive.file
drive.appfolder
“drive.appfolder” needs a small amount of extra code within rclone - the others shouldn’t.
Thanks, Nick, I will make an issue on GitHub, but probably not until I’m back in the office on Monday, apologies for the delay. And I agree it makes very good sense to roll up all the GDrive scope issues to allow users to choose among those that have been flagged by rclone users as desirable.
Responding to your direct question, how the drive.file scope differs from drive … as you found on the GDrive OAuth documentation page, drive.file permits “Per-file access to files created or opened by the app. File authorization is granted on a per-user basis and is revoked when the user deauthorizes the app.”
What this means is that rclone can read/view only those files and folders it creates. So if – before using rclone to move data to my GDrive, which is already populated via the GDrive web interface with many files and folders (which I can see and manipulate as permissions allow in the web interface) – I use an instance of rclone authorized at drive.file scope, I see nothing via rclone, e.g., like this for an rclone config entry with the name access-test –
$ ./rclone lsd access-test:
$ ./rclone ls access-test:
$
Then, if I use rclone to upload a file called testfile.txt from a local directory /home/steve/Documents/bkp, say into a folder that I’m creating with rclone to make it interesting, I can see the file and folder just created, but only those:
Of additional interest when rclone has the drive.file scope –
A new folder created by rclone with the same display name as a pre-existing folder created in the GDrive web interface will result in a new folder of the same display name (but a different id & so a different URL)
A new file created or uploaded via the GDrive web interface to a folder rclone created will be visible in the web interface, but will not be visible to / readable by rclone
Is that the level of description you’re looking for?
I have created a GitHub issue to request this feature, issue 2000.
I am not aware of any way to grant rclone access to an existing file/folder (i.e., one not created by rclone) when in drive.file access scope. That’s not an authoritative answer: I just haven’t seen any way of doing this. Nothing in the GDrive permissions dialog suggests that there’s a separate grant of permission to the rclone app on an rclone-created GDrive file, so I can say that clues to how one might do this aren’t evident in the GDrive web interface.
