Cache - is there a way to speedup writes to cache-tmp-upload-path?

Help and Support
1

I’ve just started using cache and I’m trying to copy my files from my unRAID array to my mount. I’m trying to use the cache-tmp-upload-path feature, but it’s taking an eternity to copy files to the cache-tmp-upload-path - I’m getting around 2MB/s for transfers.

Is there a way to speed this up? My mount path is /mnt/disks/google_media and my cache-tmp-upload-path is /mnt/user/rclone_upload although in reality this is /mnt/disk2/rclone_upload as I’ve set this user share to only use disk 2.

copying from mnt/diskx/FOLDER to /mnt/disks/google_media/FOLDER is slow even if I do /mnt/disk2/FOLDER i.e. the same drive as the cache-tmp-upload-path (/mnt/disk2/rclone_upload)

What I’m doing at the moment is doing a manual move to bulk upload files using a upload_gdrive_media: crypt I’ve setup on the same remote, but Ideally I’d rather just use the background upload feature so that all files are visible in the cache quicker, even if they haven’t actually uploaded yet.

Here’s my rclone config - please shout if there’s anything else in there ‘daft’ as I’m new. My online connection is 200/200, so let me know if my settings aren’t optimised.

Thanks in advance

rclone mount --config=/root/.config/rclone/rclone.conf --allow-other --dir-cache-time=160h --cache-chunk-size=10M --cache-info-age=168h --cache-workers=6 --cache-tmp-upload-path /mnt/user/rclone_upload --cache-tmp-wait-time 1m --buffer-size 0M --log-level INFO gdrive_media: /mnt/disks/google_media &

[gdrive]
type = drive
client_id = REDACTED.apps.googleusercontent.com
client_secret = REDACTED
scope = drive
root_folder_id = 
service_account_file = 
token = {"access_token":"REDACTED":"REDACTED","expiry":"2018-06-14T19:04:01.421796372+01:00"}

[cache]
type = cache
remote = gdrive:crypt
plex_url = http://192.168.30.90:32400
plex_username = Binson_Buzz
plex_password = REDACTED
chunk_size = 10M
info_age = 48h
chunk_total_size = 32G
plex_token = REDACTED

[gdrive_media]
type = crypt
remote = cache:
filename_encryption = standard
directory_name_encryption = true
password = REDACTED
password2 = REDACTED

[upload_gdrive_media]
type = crypt
remote = gdrive:crypt
filename_encryption = standard
directory_name_encryption = true
password = REDACTED
password2 = REDACTED
2

I had the same problem but then I increased the --cache-tmp-wait-time 1m to 24h (have a lot of space) and it improved immensely.

1 Like
3

Hmm I’d tried 30 mins as I thought it might be that cache is trying to upload files at the same time as new files are being added to the cache, but it didn’t help.

Thanks - I’ll give 24h a go, but doesn’t that just mean you get a day from startup to transfer files, and then if there’s some kind of cache ‘congestion’ you’ll just get bottlenecks for any transfers that occur after the first day when the backed up files are finally uploaded? i.e. you’re buggered on the 2nd day?

4

Yes I would have thought that too. If I understand the original post correctly, we’re only interested here in the speed of copying from an external drive to the tmp cache folder on the local drive. ie. noting to do with uploading and the --cache-tmp-wait-time setting.

I also noticed that copying from my NAS on the LAN to the cache-tmp-upload-path on my local drive seems much slower than copying from the same NAS to other local folders on the same local drive.