What is the problem you are having with rclone?
Rclone is producing an I/O error when trying to use an rclone mount directory as a target for an application backup (couchbase).
Run the command 'rclone version' and share the full output of the command.
rclone v1.59.1
- os/version: ubuntu 18.04 (64 bit)
- os/kernel: 5.4.0-1086-azure (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.18.5
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Azure Blob Container
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
Rclone mount:
/usr/bin/rclone mount azure-blob:couchbase-stg-backup-test \
/mnt/azure-blob \
--allow-other \
--attr-timeout 1000h \
--buffer-size 64M \
--dir-cache-time 10000h \
--log-level INFO \
--log-file /tmp/rclone.log \
--poll-interval 30s \
--use-mmap \
--cache-dir=/datadisk/rclone-cache/ \
--vfs-cache-mode writes \
--transfers 100 \
--timeout 1h \
--umask 002
I know the polling flag is useless here. I just forgot to remove it from the systemd service.
The actual command causing the error is a couchbase backup:
/opt/couchbase/bin/cbbackupmgr backup --cluster "http://localhost:8091" --archive /mnt/azure-blob/backups/ --repo 2022-08-25 --username *** --password *** --resume
And the error produced by couchbase:
Error backing up cluster: input/output error
The rclone config contents with secrets removed.
[azure-blob]
type = azureblob
account = ***
key = ***
A log from the command with the -vv
flag
2022/08/25 20:33:32 INFO : Azure container couchbase-stg-backup-test: poll-interval is not supported by this remote
2022/08/25 20:33:32 INFO : vfs cache: cleaned: objects 96 (was 96) in use 0, to upload 0, uploading 0, total size 2.686Mi (was 2.686Mi)
2022/08/25 20:33:42 INFO : backups/lock.lk: vfs cache: queuing for upload in 5s
2022/08/25 20:33:42 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-0: vfs cache: queuing for upload in 5s
2022/08/25 20:33:42 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-0: vfs cache: renamed in cache to "backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/.collection_manifest"
2022/08/25 20:33:42 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/.collection_manifest: vfs cache: queuing for upload in 5s
2022/08/25 20:33:43 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-1: vfs cache: queuing for upload in 5s
2022/08/25 20:33:43 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-1: vfs cache: renamed in cache to "backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/failoverlog_321.fol"
2022/08/25 20:33:43 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-2: vfs cache: queuing for upload in 5s
2022/08/25 20:33:43 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-2: vfs cache: renamed in cache to "backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/failoverlog_343.fol"
2022/08/25 20:33:43 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-3: vfs cache: queuing for upload in 5s
2022/08/25 20:33:43 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-3: vfs cache: renamed in cache to "backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/failoverlog_543.fol"
2022/08/25 20:33:43 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-4: vfs cache: queuing for upload in 5s
2022/08/25 20:33:43 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/data_343.rift.0: vfs cache: queuing for upload in 5s
2022/08/25 20:33:43 ERROR : IO error: can't open file - writer failed
2022/08/25 20:33:43 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-4: vfs cache: renamed in cache to "backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/failoverlog_563.fol"
2022/08/25 20:33:43 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-5: vfs cache: queuing for upload in 5s
2022/08/25 20:33:43 INFO : backups/lock.lk: vfs cache: removed cache file as file deleted
2022/08/25 20:33:43 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/index_321.sqlite.0: vfs cache: queuing for upload in 5s
2022/08/25 20:33:43 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/index_343.sqlite.0: vfs cache: queuing for upload in 5s
2022/08/25 20:33:43 INFO : backups/logs/stats/disk/2022-08-25-backup-1661459622: vfs cache: queuing for upload in 5s
2022/08/25 20:33:43 INFO : backups/logs/stats/net/2022-08-25-backup-1661459622: vfs cache: queuing for upload in 5s
2022/08/25 20:33:43 INFO : backups/logs/stats/cpu/2022-08-25-backup-1661459622: vfs cache: queuing for upload in 5s
2022/08/25 20:33:43 INFO : backups/logs/backup-0.log: vfs cache: queuing for upload in 5s
2022/08/25 20:33:47 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/.collection_manifest: Copied (new)
2022/08/25 20:33:47 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/.collection_manifest: vfs cache: upload succeeded try #1
2022/08/25 20:33:48 INFO : backups/logs/stats/disk/2022-08-25-backup-1661459622: Copied (new)
2022/08/25 20:33:48 INFO : backups/logs/stats/disk/2022-08-25-backup-1661459622: vfs cache: upload succeeded try #1
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/failoverlog_321.fol: Copied (new)
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/failoverlog_321.fol: vfs cache: upload succeeded try #1
2022/08/25 20:33:48 INFO : backups/logs/stats/net/2022-08-25-backup-1661459622: Copied (new)
2022/08/25 20:33:48 INFO : backups/logs/stats/net/2022-08-25-backup-1661459622: vfs cache: upload succeeded try #1
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/data_343.rift.0: Copied (new)
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/data_343.rift.0: vfs cache: upload succeeded try #1
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/index_321.sqlite.0: Copied (new)
2022/08/25 20:33:48 INFO : backups/logs/stats/cpu/2022-08-25-backup-1661459622: Copied (new)
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/index_321.sqlite.0: vfs cache: upload succeeded try #1
2022/08/25 20:33:48 INFO : backups/logs/stats/cpu/2022-08-25-backup-1661459622: vfs cache: upload succeeded try #1
2022/08/25 20:33:48 INFO : backups/logs/backup-0.log: Copied (replaced existing)
2022/08/25 20:33:48 INFO : backups/logs/backup-0.log: vfs cache: upload succeeded try #1
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/failoverlog_543.fol: Copied (new)
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/failoverlog_563.fol: Copied (new)
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/failoverlog_543.fol: vfs cache: upload succeeded try #1
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-5: Copied (new)
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/failoverlog_563.fol: vfs cache: upload succeeded try #1
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/temp-0-5: vfs cache: upload succeeded try #1
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/failoverlog_343.fol: Copied (new)
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/failoverlog_343.fol: vfs cache: upload succeeded try #1
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/index_343.sqlite.0: Copied (new)
2022/08/25 20:33:48 INFO : backups/2022-08-25/2022-08-25T20_06_46.73886858Z/global-ad989971c12b05a4b8b015ae28e152b5/data/index_343.sqlite.0: vfs cache: upload succeeded try #1
DEGUG log is huge.
rclone.log (1.1 MB)