What is the problem you are having with rclone?
Rclone Docker Volume Plugin on WSL2 client with Synology NAS to host volume
Run the command 'rclone version' and share the full output of the command.
well... it's a docker plugin, docker plugin inspect rclone does not reveal a version number. Installed as a managed plugin on a Windows 11 machine in WSL2 with docker 20.10.21 installed directly in Ubuntu 22.04.1 LTS for WSL (not using Windows Docker Desktop in this setup as another machine with that installed will not even install the plugin).
Plugin installs successfully with:
docker plugin install rclone/docker-volume-rclone:amd64 --alias rclone --grant-all-permissions args="-v --allow-other --vfs-cache-mode=writes" config=/home/dude/rclone/config cache=/home/dude/rclone/cache
but am not able to create a volume on the local NAS with sftp from the WSL2 client:
docker volume create test_data -d rclone -o type=sftp -o sftp-host=192.168.3.4 -o --sftp-user=dude -o --sftp-key-file=/home/dude/.ssh/id_rsa -o --sftp-known-hosts-file=/home/dude/.ssh/known_hosts
results in:
Error response from daemon: create media: VolumeDriver.Create: couldn't parse known_hosts_file: open /home/dude/.ssh/known_hosts: no such file or directory
Tried running with and without sudo, same result.
SFTP & SSH using id_rsa keyfile (passwordless) both connect successfully when run manually and the NAS is in the known_hosts. I get the same error message "no such file or directory" for id_rsa when -o --sftp-known-hosts-file=/home/dude/.ssh/known_hosts is omitted in the volume create command as well. I've tried placing known_hosts & id_rsa in numerous different directories, still can't be found per the plugin. Even tried giving them the dreaded chmod 777 to no avail. Also tried creating a volume using an sftp config in rclone.conf but get same error.
Not really sure where I could look for helpful logs. /var/log/syslog just registers the same error message as when the volume create command is run.
Is this just not possible on WSL2? I've been banging my head against this for a week & it took a lot of troubleshooting just to get to this point. Can't find much discussion about this plugin with my google fu. Any suggestions welcome, thanks.
Making progress. It finally clicked with me that the volume plugin is essentially just like any other docker container. When I type the volume create command the plugin is looking within it's own data structure for id_rsa & known_hosts (ie container cache & config are bind mounted to the host folders). I coped those files from ~./shh over to my config folder specified on install (config=/home/dude/rclone/config).
Volume create is successful with this command:
docker volume create test_data -d rclone -o type=sftp -o sftp-host=192.168.3.4 -o --sftp-user=dude -o --sftp-key-file=/data/config/id_rsa -o --sftp-known-hosts-file=/data/config/known_hosts
$ docker volume ls
DRIVER VOLUME NAME
rclone:latest test_data
Haven't done much testing yet but we'll see how it goes. Took a lot of effort to finally at least create a basic volume with the plugin. Maybe I will write a guide or something eventually as I found very little info on creating volumes on docker on wsl2 using rclone volume plugin.
1 Like
Trying to get my actual intended docker environment set up this morning and have some questions. (To be clear, this is run on a Windows 11 machine with docker installed directly in Ubuntu WSL2. When run with Windows Docker Desktop installed even with WSL 2 back end enable the rclone volume plugin WILL NOT install).
As above, this command does, create a persistent sftp volume on the NAS:
docker volume create test_data -d rclone -o type=sftp -o sftp-host=192.168.3.4 -o --sftp-user=dude -o --sftp-key-file=/data/config/id_rsa -o --sftp-known-hosts-file=/data/config/known_hosts
$ docker volume ls
DRIVER VOLUME NAME
rclone:latest test_data
Copied an rclone create SFTP config into the plugin config folder (/home/dude/rclone/config) and can successfully create a volume as well with:
docker volume create test_data -d rclone -o remote=server: -o vfs-cache-mode=writes
This of course mounts the "dude" user's sftp "/" root folder on my Synology NAS, which contains "docker", "home", and "homes" directories. I can mount the test_data sftp volume on my WSL2 docker client with an Ubuntu test container, see those folders, and can touch test.txt which persists when the volume is removed & recreated. However, Synology File Station and ssh'ing into the NAS don't show test.txt with ls -al no matter where I place it. But again, test.txt does persist across volume rm & create. I'm assuming this has something to do with docker itself doing some funny business with the file system but I have no idea. Doesn't really matter though if the sftp volume contents persists.
Contents of rclone.conf
[server]
type = sftp
host = 192.168.3.4
user = dude
key_file = ${RCLONE_CONFIG_DIR}/id_rsa
known_hosts = ${RCLONE_CONFIG_DIR}/known_hosts
shell_type = unix
md5sum_command = md5sum
sha1sum_command = sha1sum
I guess my question is, what should I put for remote folder when creating the volume(s)?
-
remote=server:/docker/volumes/test_data ?
-
remote=server:/home/ ?
-
remote=server:/home/Drive/Docker/volumes/test_data ?
etc, etc, etc
Does it matter? I'm assuming I should indicate a specific folder for each volume either way. Going to play with it a bit more later today. Any suggestions welcome.