First, rclone is an awsome tool I'm using since years now for different purpose, very stable. Now I need some guidance/advice in caching strategy.
I have a 8 mechanical HDD array in a Unraid server.
I have a 2T SSD hosting docker and VM, and I was using Unraid cache system to store only new files, with a script moving to array when SSD was 90% full. SSD path is
I'm not using external cloud, everything are local on the same machine .
I have 40GB memory, only about 8GB are used.
I use plex to stream to several devices on LAN (gigabit ethernet) or WAN (gigabit fiber internet), and also seed torrents with transmission.
It is on a Unraid machine, process below is handled by OS:
Data are merged from all drives to
SSD is mounted to
My apps access to
/mnt/user/data which is a merge of SSD and Array: system pick in SSD, if file is not there it picks in Array, if duplicate it use SSD's file.
What are you trying to achieve with rclone?
I'd like to use SSD as a working drive, and HDD for long term storage/backup.
My goal is then to cache full file on the SSD (unlimited time in a FIFO manner using 1T ) so apps are only accessing the SSD. Docker or VM volumes for other apps are already on the SSD.
I need advice about the goog method/parameters to use to cache from array to SSD using rclone (or using something else, maybe rclone is not the best for this job, but that's my best bet at the moment)
Now I'm testing
rclone cache or
rclone local remotes associated with
rclone mount with different parameters.
Either rclone mounting with local remote or rclone cached remote work perfectly and files are read/writen from/to the SSD, but I'd rather want full file and transparent caching (no wait time for plex and transmission) and it seems by listing mounting point it's not the case.
What is your rclone version (output from
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Linux 4.19.98-Unraid #1 SMP Sun Feb 2 22:45:43 GMT 2020 x86_64 AMD Ryzen 7 3700X 8-Core Processor AuthenticAMD GNU/Linux
Which cloud storage system are you using? (eg Google Drive)
The command you were trying to run (eg
rclone copy /tmp remote:tmp)
[data-cache] type = cache remote = local:/mnt/user0/data plex_url = http://192.168.1.20:32400 plex_username = *REDACTED* plex_password = *REDACTED* chunk_size = 10M info_age = 2d chunk_total_size = 1T db_path = /mnt/cache/rclone/cache-backend chunk_path = /mnt/cache/rclone/cache-backend workers = 10 chunk_no_memory = true writes = true plex_token = *REDACTED* [local] type = local
mntpoint4="/mnt/cache/data" remoteshare4="data-cache:" #(or local: local path) rclone mount --max-read-ahead 1024k --uid 99 --gid 100 --allow-other --vfs-cache-mode write $remoteshare4 $mntpoint4 &
Command method 2:
mntpoint5="/mnt/cache/data" remoteshare5="local:/mnt/user0/data" rclone mount --max-read-ahead 1024k --uid 99 --gid 100 --allow-other --vfs-cache-mode full --vfs-cache-max-age 48h --vfs-cache-max-size 1T $remoteshare5 $mntpoint5 &
Thank you for your help and guidance