Cache Offline Uploading - Trigger a "Push"?


Loving the offline uploading feature, makes a very hand write cache. Question: I want to set my tmp_wait_time to something high like 24 hours, however I'm concerned that if a lot of writes occur I will fill out my disk space. I want to write a watcher that says "If the disk space gets to 80%, flush out some of the rclone files waiting in the upload buffer"

The files not being on my remote for a long time isn't a big deal, I'm basically using it as a tiered storage.

Is there some way via RC or something to tell it to "flush" the upload buffer and send everything? Even better, pick them out on a per file basis? Maybe set tmp_wait_time temporarily to 5 seconds or something?


Are we talking about the cache backend here or did Nick get around to implementing offline uploading for the VFS when I wasn't looking?

Cache backend. These parameters:
tmp_upload_path = /tmp/rclonecache/upload
tmp_wait_time = 60m

Another thought on this: If I rclone mv the files out of the upload buffer to the remote backend ahead of time, will rclone freak out that the file isn't there when it comes time to upload? I could just do this instead selectively and that works, and that's what the operation does anyways.

As far as I know, rclone does not currently keep any lists of what is "supposed" to be uploaded. It essentially just does a move on a timer - so yes - you should be fine to shortcut this on your own by an external script that just triggers anytime you hit a certain size.

And as far as the RC goes - it does have some cache functions, indicating that the system to send messages to the cache-backend is in place. However it to the best of my knowledge does not have the function you are asking for. It would have to be implemented.

Thanks for the thoughts. I'll POC the "move" method and report back

--max-transfer (in case you were not aware of it already) is probably useful to limit the scope of each triggered event. You can always just let it trigger again if more needs to be moved.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.