What is the problem you are having with rclone?
Hi, I'm using rclone as an SSD cache for my HDD array (yeah, it's a weird way to use rclone), it works great! But I'm having a little issue with big files.
My setup is: rclone mount -> samba -> my desktop PC to access the files
The operating system is Unraid
The parameters I'm using are:
rclone mount --cache-dir localcache/ --vfs-cache-mode full --vfs-read-ahead 128M --allow-other /mnt/disk1/ /mnt/rclonecache/
When I open a 1GB file, it hangs on "openPending" for about 10 seconds before starting to read the file (localcache directory stays empty, so it's not even pre-caching it).
The issue is only with big files, small files loads up almost instantly.
I've tried several options, including changing the buffer size, and the --no-checksum flag.
Any help would be appreciated, thanks in advance!
A log from the command with the -vv
flag
2022/03/21 12:28:21 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac: Open: flags=OpenReadOnly+OpenNonblock
2022/03/21 12:28:21 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac: Open: flags=O_RDONLY|0x800
2022/03/21 12:28:21 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac: newRWFileHandle:
2022/03/21 12:28:21 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac: >newRWFileHandle: err=<nil>
2022/03/21 12:28:21 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac: >Open: fd=musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac (rw), err=<nil>
2022/03/21 12:28:21 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac: >Open: fh=&{musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac (rw)}, err=<nil>
2022/03/21 12:28:22 DEBUG : &{musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac (rw)}: Read: len=131072, offset=0
2022/03/21 12:28:22 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac(0xc0008c15c0): _readAt: size=131072, off=0
2022/03/21 12:28:22 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac(0xc0008c15c0): openPending:
2022/03/21 12:28:32 DEBUG : /: Attr:
2022/03/21 12:28:32 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxrwxrwx, err=<nil>
2022/03/21 12:28:32 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac: Attr:
2022/03/21 12:28:34 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac: vfs cache: checking remote fingerprint "1983554435,2019-02-08 16:16:26.2152148 +0000 UTC,3b0065b464608a6855a25e7af6cac0ff" against cached fingerprint "1983554435,2019-02-08 16:16:26.2152148 +0000 UTC,3b0065b464608a6855a25e7af6cac0ff"
2022/03/21 12:28:34 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac: vfs cache: truncate to size=1983554435
2022/03/21 12:28:34 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac: >Attr: a=valid=1s ino=0 size=1983554435 mode=-rw-rw-rw-, err=<nil>
2022/03/21 12:28:34 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal: Added virtual directory entry vAddFile: "Sempiternal.flac"
2022/03/21 12:28:34 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac(0xc0008c15c0): >openPending: err=<nil>
2022/03/21 12:28:34 DEBUG : vfs cache: looking for range={Pos:0 Size:131072} in [{Pos:0 Size:156229632} {Pos:1983524864 Size:29571}] - present true
2022/03/21 12:28:34 DEBUG : musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac(0xc0008c15c0): >_readAt: n=131072, err=<nil>
2022/03/21 12:28:34 DEBUG : &{musica/musica/Bring Me The Horizon/Sempiternal/Sempiternal.flac (rw)}: >Read: read=131072, err=<nil>
...
Run the command 'rclone version' and share the full output of the command.
rclone v1.58.0
- os/version: slackware 14.2+ (64 bit)
- os/kernel: 5.10.28-Unraid (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.17.8
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
none, I'm using local storage
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rclone mount --cache-dir localcache/ --vfs-cache-mode full --vfs-read-ahead 128M --allow-other /mnt/disk1/ /mnt/rclonecache/
The rclone config contents with secrets removed.
Haven't used `rclone config`, so the file is empty.