What is the problem you are having with rclone?
After some time of running container, docker volume plugin (rclone serve docker) when using VFS cache throws exception fatal error: concurrent map writes. After this content of volume is not available anymore, so I have to recreate volume (and container which depends on it).
Run the command 'rclone version' and share the full output of the command.
$ sudo runc --root /run/docker/runtime-runc/plugins.moby exec $(docker plugin list --no-trunc | awk '/rclone/{print$1}') rclone version
rclone v1.64.0
- os/version: alpine 3.18.3 (64 bit)
- os/kernel: 6.2.0-1011-azure (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.21.1
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
azureblob + crypt
The command you were trying to run (eg rclone copy /tmp remote:tmp)
docker plugin set rclone args="--log-level DEBUG --log-file /var/log/rclone.log --allow-other --buffer-size 32Mi --dir-cache-time 1d --fast-list --links --poll-interval 1h --read-only --vfs-cache-mode full --vfs-cache-max-age 1w --vfs-cache-max-size 0 --vfs-cache-min-free-space 10G --vfs-cache-poll-interval 1m --vfs-write-back 1m --vfs-fast-fingerprint --vfs-read-ahead 64Mi"
services:
plex:
volumes:
- rclone_azure_crypt_music:/media/music:ro
volumes:
rclone_azure_crypt_music:
driver: rclone
driver_opts:
remote: "azure-crypt:Music"
The rclone config contents with secrets removed.
[azure]
type = azureblob
account = *****
key = *****
directory_markers = true
[azure-crypt]
type = crypt
remote = azure:crypt
password = *****
password2 = *****
filename_encoding = base64
A log from the command with the -vv flag
Logs from plugin
sudo runc --root /run/docker/runtime-runc/plugins.moby exec --tty $(docker plugin list --no-trunc | awk '/rclone/{print$1}') cat /var/log/rclone.log
fatal error: concurrent map writes
goroutine 396926 [running]:
github.com/rclone/rclone/backend/azureblob.(*Object).setMetadata(0xc00b947c70, 0xc006821e60)
/go/src/github.com/rclone/rclone/backend/azureblob/azureblob.go:1631 +0x178
github.com/rclone/rclone/backend/azureblob.(*Object).decodeMetaDataFromDownloadResponse(0xc00b947c70, 0xc00bd0b600)
/go/src/github.com/rclone/rclone/backend/azureblob/azureblob.go:1736 +0x1a9
github.com/rclone/rclone/backend/azureblob.(*Object).Open(0xc00b947c70, {0x2760318?, 0x397f1c0}, {0xc00be37be0, 0x2, 0x2})
/go/src/github.com/rclone/rclone/backend/azureblob/azureblob.go:1935 +0x485
github.com/rclone/rclone/backend/crypt.(*Object).Open.func1({0x2760318, 0x397f1c0}, 0x42a42c0, 0x10010?)
/go/src/github.com/rclone/rclone/backend/crypt/crypt.go:974 +0x174
github.com/rclone/rclone/backend/crypt.(*decrypter).RangeSeek(0xc00be39320, {0x2760318, 0x397f1c0}, 0x42ac000, 0x30?, 0x8000000)
/go/src/github.com/rclone/rclone/backend/crypt/cipher.go:1009 +0x376
github.com/rclone/rclone/backend/crypt.(*Cipher).newDecrypterSeek(0x0?, {0x2760318, 0x397f1c0}, 0xc0065e9380, 0x42ac000, 0x8000000)
/go/src/github.com/rclone/rclone/backend/crypt/cipher.go:849 +0x19d
github.com/rclone/rclone/backend/crypt.(*Cipher).DecryptDataSeek(0xc0027a1b78?, {0x2760318?, 0x397f1c0?}, 0x4105c5?, 0x1db5fe0?, 0x20?)
/go/src/github.com/rclone/rclone/backend/crypt/cipher.go:1113 +0x1d
github.com/rclone/rclone/backend/crypt.(*Object).Open(0xc0027a1b78, {0x2760318, 0x397f1c0}, {0xc00bddb9e0?, 0x2, 0x2?})
/go/src/github.com/rclone/rclone/backend/crypt/crypt.go:960 +0x30b
github.com/rclone/rclone/fs/chunkedreader.(*ChunkedReader).openRange(0xc00c696690)
/go/src/github.com/rclone/rclone/fs/chunkedreader/chunkedreader.go:225 +0x4b2
github.com/rclone/rclone/fs/chunkedreader.(*ChunkedReader).Read(0xc00c696690, {0xc017964000?, 0xc007df0601?, 0xc00c314f10?})
/go/src/github.com/rclone/rclone/fs/chunkedreader/chunkedreader.go:92 +0x315
github.com/rclone/rclone/lib/readers.ReadFill({0x273f360, 0xc00c696690}, {0xc017964000, 0x1000, 0x100000})
/go/src/github.com/rclone/rclone/lib/readers/readfill.go:14 +0x63
github.com/rclone/rclone/fs/asyncreader.(*buffer).read(0xc0065e8f90, {0x273f360?, 0xc00c696690?})
/go/src/github.com/rclone/rclone/fs/asyncreader/asyncreader.go:351 +0x3a
github.com/rclone/rclone/fs/asyncreader.(*AsyncReader).init.func1()
/go/src/github.com/rclone/rclone/fs/asyncreader/asyncreader.go:95 +0x153
created by github.com/rclone/rclone/fs/asyncreader.(*AsyncReader).init in goroutine 396911
/go/src/github.com/rclone/rclone/fs/asyncreader/asyncreader.go:83 +0x1c5
Full log: rclone logs - Pastebin.com
Mounted volume not accesible anymore:
docker exec -it plex ls -lah /media/music
ls: cannot access '/media/music': Transport endpoint is not connected
Notes
Plugin without configured VFS cache arguments works fine.
I found similar, fixed in 1.51 bug, but in obsolete 'cache' provider.
Maybe the same change should be implemented in VFS code?
