How to use google drive as docker volume

Help and Support
1

What is the problem you are having with rclone?

I want to make some software use google cloud as storage. These software come packaged as docker images, where you can attach a volume to it, and it will store the data there.

I created a remote in rclone called GDrive and mounted it to /home/me/GDrive.

But if I simply try to pass some directory inside it as volume, I get an error as follows

docker run -i -v ~/GDrive/SoftwareData/archivebox:/data -p 8000:8000 archivebox/archivebox
docker: Error response from daemon: error while creating mount source path '/home/me/GDrive/SoftwareData/archivebox': mkdir /home/me/GDrive: file exists.

So apparently this won't work.

Another option I considered is to create a docker volume using the docker volume plugin for rclone. But I am not able to understand the command properly.

I tried,

docker volume create myvol1 -d rclone -o remote=GDrive:/home/me/GDrive/SoftwareData
Error response from daemon: create myvol1: VolumeDriver.Create: didn't find section in config file

So this must be wrong too. Where am I going wrong?

What is your rclone version (output from rclone version)

rclone v1.56.2

  • os/version: arch 21.1.5 (64 bit)
  • os/kernel: 5.13.19-2-MANJARO (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.17.1
  • go/linking: dynamic
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

docker volume create myvol1 -d rclone -o remote=GDrive:/home/me/GDrive/SoftwareData

The rclone config contents with secrets removed.

[OneDrive]
type = onedrive
token = {"access_token":"t1","token_type":"Bearer","refresh_token":"t2","expiry":"2020-05-27T21:23:53.05130635+05:30"}
drive_id = id1
drive_type = business

[GDrive]
type = drive
client_id = someid.apps.googleusercontent.com
client_secret = somes3cr3t
scope = drive
token = {"access_token":"sometok3n","token_type":"Bearer","refresh_token":"anothertoken","expiry":"2021-10-16T22:32:17.112603611+05:30"}
team_drive =

A log from the command with the -vv flag

It's a docker command. Not sure how to pass -vv

2

I also tried the following

docker volume create myvol1 -d rclone -o remote=GDrive

Does create a volume, but it doesn't have any of the files from the remote. So most likely it's doing something entirely different. I am unable to figure out what the remote option is expected to take. It seems to me from the example at Docker Volume Plugin that it's the name of the remote from the config file.

3

It's not name, it's "remote path" and should have a colon as such, namely RemoteName:subdir, or just RemoteName: if initial subdir is enough.

Without a colon, it means something stupid - refers to a relative directory inside the plugin.

You want to recreate your volume with colon in remote:

docker volume rm myvol1
docker volume create myvol1 -d rclone -o remote=GDrive:
4

@albertony
Using local remotes (without a colon) in connection with docker plugin is so useless (who would want to mount a directory from plugin container?) that I'm thinking of making colon a requirement and return error otherwise.

5

You are triple wrong:

  1. Docker plugin needs a config file in a special location, independent from your user's rclone config.
  2. Your GDrive does not have a /home/me/GDrive/SoftwareData directory you requesting
  3. You didn't read the manual where it's all explained

.....

Ahh, one more.....

You wanted to use a remote in containers. Docker gives you docker run -v ... on command line and volumes in compose right for that.
Now you mount it on host. Why?

Why?

6

Sounds like a good idea!
I've noticed there are quite a few "support" cases due to missing colons in general... Not sure how to avoid it, but at least for the container use case it makes sense to treat it as an error.

1 Like
7

Thanks, it makes sense now!

My brain was under the impression that the config file is the same as the default one! Note for future readers: Skimming docs is not good.

As for why I mounted it, Initially I thought I could just mount some directory inside remote to docker, but that didn't work.

8

One more clarification question,

docker volume with rclone will only allow root:root to write, right? So anything inside the container that wants to write files must do so as root user. If I understand correctly, this is because google drive doesn't understand file permission the same way as unix does.

9

Use uid=X, gid=X or allow_other - rclone serve docker

10

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.