Docker volume gone after reboot

What is the problem you are having with rclone?

Using the Docker Volume Plugin with Sftp like in the documentation. Creating a volume and using it in a container works as described. The volume is usable and can be checked by using "docker volume list". After reboot the volume is gone and I need to run "docker volume create" again.

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.2
- os/version: ubuntu 20.04 (64 bit)
- os/kernel: 5.4.0-125-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.18.6
- go/linking: static
- go/tags: none

The command you were trying to run (eg rclone copy /tmp remote:tmp)

docker volume create local-sftp -d rclone -o type=sftp -o sftp-host=localhost -o sftp-port=3322 -o sftp-user=mount-playlist -o sftp-pass=abcdzDRKW0nDymRJNtKIc32J8g-y4BPv40GTAjaiFmnqwftcoekqVlr3yG6-SqYbT98IQoF0QHDgX-Su61XhppK98umLN5qI8SNTLGKvY5Y -o allow-other=true

The rclone config contents with secrets removed.

user@example:/$ docker plugin inspect rclone
[
    {
        "Config": {
            "Args": {
                "Description": "",
                "Name": "args",
                "Settable": [
                    "value"
                ],
                "Value": []
            },
            "Description": "Rclone volume plugin for Docker",
            "DockerVersion": "20.10.17+azure-3",
            "Documentation": "https://rclone.org/docker",
            "Entrypoint": [
                "rclone",
                "serve",
                "docker"
            ],
            "Env": [
                {
                    "Description": "",
                    "Name": "RCLONE_VERBOSE",
                    "Settable": [
                        "value"
                    ],
                    "Value": "0"
                },
                {
                    "Description": "",
                    "Name": "RCLONE_CONFIG",
                    "Settable": null,
                    "Value": "/data/config/rclone.conf"
                },
                {
                    "Description": "",
                    "Name": "RCLONE_CACHE_DIR",
                    "Settable": null,
                    "Value": "/data/cache"
                },
                {
                    "Description": "",
                    "Name": "RCLONE_BASE_DIR",
                    "Settable": null,
                    "Value": "/mnt"
                },
                {
                    "Description": "",
                    "Name": "HTTP_PROXY",
                    "Settable": [
                        "value"
                    ],
                    "Value": ""
                },
                {
                    "Description": "",
                    "Name": "HTTPS_PROXY",
                    "Settable": [
                        "value"
                    ],
                    "Value": ""
                },
                {
                    "Description": "",
                    "Name": "NO_PROXY",
                    "Settable": [
                        "value"
                    ],
                    "Value": ""
                }
            ],
            "Interface": {
                "Socket": "rclone.sock",
                "Types": [
                    "docker.volumedriver/1.0"
                ]
            },
            "IpcHost": false,
            "Linux": {
                "AllowAllDevices": false,
                "Capabilities": [
                    "CAP_SYS_ADMIN"
                ],
                "Devices": [
                    {
                        "Description": "",
                        "Name": "",
                        "Path": "/dev/fuse",
                        "Settable": null
                    }
                ]
            },
            "Mounts": [
                {
                    "Description": "",
                    "Destination": "/data/config",
                    "Name": "config",
                    "Options": [
                        "rbind"
                    ],
                    "Settable": [
                        "source"
                    ],
                    "Source": "/var/lib/docker-plugins/rclone/config",
                    "Type": "bind"
                },
                {
                    "Description": "",
                    "Destination": "/data/cache",
                    "Name": "cache",
                    "Options": [
                        "rbind"
                    ],
                    "Settable": [
                        "source"
                    ],
                    "Source": "/var/lib/docker-plugins/rclone/cache",
                    "Type": "bind"
                }
            ],
            "Network": {
                "Type": "host"
            },
            "PidHost": false,
            "PropagatedMount": "/mnt",
            "User": {},
            "WorkDir": "/data",
            "rootfs": {
                "diff_ids": [
                    "sha256:abcde07cc7f702ef4a01cbc02ffabad3ffd276431e683201767ee202addd6903"
                ],
                "type": "layers"
            }
        },
        "Enabled": true,
        "Id": "abcd588fcedf5d09676d83a5f3f4dbf693a643b1ad8df9db52b9058126281d3a",
        "Name": "rclone:latest",
        "PluginReference": "docker.io/rclone/docker-volume-rclone:amd64",
        "Settings": {
            "Args": [
                "-v"
            ],
            "Devices": [
                {
                    "Description": "",
                    "Name": "",
                    "Path": "/dev/fuse",
                    "Settable": null
                }
            ],
            "Env": [
                "RCLONE_VERBOSE=0",
                "RCLONE_CONFIG=/data/config/rclone.conf",
                "RCLONE_CACHE_DIR=/data/cache",
                "RCLONE_BASE_DIR=/mnt",
                "HTTP_PROXY=",
                "HTTPS_PROXY=",
                "NO_PROXY="
            ],
            "Mounts": [
                {
                    "Description": "",
                    "Destination": "/data/config",
                    "Name": "config",
                    "Options": [
                        "rbind"
                    ],
                    "Settable": [
                        "source"
                    ],
                    "Source": "/var/lib/docker-plugins/rclone/config",
                    "Type": "bind"
                },
                {
                    "Description": "",
                    "Destination": "/data/cache",
                    "Name": "cache",
                    "Options": [
                        "rbind"
                    ],
                    "Settable": [
                        "source"
                    ],
                    "Source": "/var/lib/docker-plugins/rclone/cache",
                    "Type": "bind"
                }
            ]
        }
    }
]

I suspect the rclone /data/config directory is inside a docker container itself. Do I need to expose that to an outside directory?

Additional information

When I go snooping in the config files I find this:

user@example:/var/lib/docker-plugins$ ls -la rclone/*/*
-rw------- 1 root root 358 Sep 19 14:04 rclone/cache/docker-plugin.state
-rw------- 1 root root   0 Sep 19 14:04 rclone/config/rclone.conf
roy@morvigor:/var/lib/docker-plugins$ sudo cat rclone/cache/docker-plugin.state
[sudo] password for user:
[{"name":"local-sftp","mountpoint":"/mnt/local-sftp","created":"2022-09-19T14:04:21.133293718Z","fs":"","type":"sftp","options":{"allow-other":"true","sftp-host":"localhost","sftp-pass":"abcdzDRKW0nDymRJNtKIc32J8g-y4BPv40GTAjaiFmnqwftcoekqVlr3yG6-SqYbT98IQoF0QHDgX-Su61XhppK98umLN5qI8SNTLGKvY5Y","sftp-port":"3322","sftp-user":"mount-playlist"},"mounts":[]}]

Some additional steps taken:

  • Updated Docker to 20.10.18
  • Updated Docker Compose to 2.11.0
  • Updated the plugin to "4cd3f97eae0c5fe7d7691a669fb1bc5103eaf92e44feb7ec451df90f1a4730df"
  • Uninstalled the plugin and reinstalled with a different config directory.
  • Installed a different volume plugin (SeaweedFS) and tested if it was persistent. (Yes)
  • Uninstalled the managed plugin and used the Systemd based plugin

Logging with the Systemd service is a lot easier then with the Plugin. The error it yields is:

DEBUG : Setting --verbose "1" from environment variable RCLONE_VERBOSE="1"
DEBUG : Setting --config "/var/lib/docker-plugins/rclone/config/rclone.conf" from environment variable RCLONE_CONFIG="/var/lib/docker-plugins/rc>
DEBUG : Setting --cache-dir "/var/lib/docker-plugins/rclone/cache" from environment variable RCLONE_CACHE_DIR="/var/lib/docker-plugins/rclone/ca>
DEBUG : rclone: Version "v1.59.2" starting with parameters ["/usr/bin/rclone" "-vv" "serve" "docker"]
DEBUG : rclone: systemd logging support activated
DEBUG : Restore state from /var/lib/docker-plugins/rclone/cache/docker-plugin.state
NOTICE: Failed to restore volume "local-sftp": cannot parse path "": can't use empty string as a path
DEBUG : Monitoring 0 volumes
INFO  : Serving systemd socket
DEBUG : /VolumeDriver.Get: Request returned error: volume not found

From the docs, the config is located:

Docker Volume Plugin (rclone.org)

I'd never used the plugin but did you create those directories per the documentation?

My volume creation syntax seemed to be wrong. It would be nice if Rclone would pick a default path like "/". I assume this to be my problem. My full working docker volume create syntax is:

docker volume create local-sftp -d rclone -o remote=:sftp,host=localhost:/ -o sftp-port=3322 -o sftp-user=mount-playlist -o sftp-pass=abcdzDRKW0nDymRJNtKIc32J8g-y4BPv40GTAjaiFmnqwftcoekqVlr3yG6-SqYbT98IQoF0QHDgX-Su61XhppK98umLN5qI8SNTLGKvY5Y -o allow-other=true

It's solved, keeping this open till someone else takes notice and closes it. Don't want to be prisoner, judge, juror and executioner.

Unfortunately our docker volume maintainer is not available at the moment.

Can you expand on this? What did you type that was wrong and what was the correct version?

Maybe we can expand the docs with this info.

Rsync gave me this error:

Failed to restore volume "local-sftp": cannot parse path "": can't use empty string as a path

That was fixed by using the shorthand syntax: remote=:sftp,host=localhost:/