Cache necessary for webdav server who works only as sync server?

Help and Support
1

What is the problem you are having with rclone?

I'm not sure if I really need a cache. Disadvantage is, that Synology cloudsync says "everything synced", but in reality the files (which shall be uploaded) are only in the local cache from the webdav server.
And I do not know if the time schedule from cloud sync is affecting the upload from local cache.

I only use the rclone webdav server as "man in the middle" to sync all files between pcloud and NAS.
Can I disable then the cache or is there still any disadvantage for my use case?

What is your rclone version (output from rclone version)

1.57.0

Which cloud storage system are you using? (eg Google Drive)

pcloud

The command you were trying to run (eg rclone copy /tmp remote:tmp)

#!/bin/sh
rclone serve webdav pcloud: --addr :8080 --user xxx --pass xxx --config /volume2/homes/Michael/.config/rclone/rclone.conf --cache-dir /volume2//Backup/rclone_cache/Michael --vfs-cache-mode writes

The rclone config contents with secrets removed.

[pcloud]
type = pcloud
hostname = eapi.pcloud.com
token = .... (removed)

A log from the command with the -vv flag

not relevant, because there is no failure

2

hi,

if you can get what you want done without the cache, then do not use it.
it depends on the application, in your case cloudsync.

i helped a follow rcloner with a similar use-case on a synbox, but the app was hyperbackup.
https://forum.rclone.org/t/rclone-docker-on-synology/27753

3

Ok. So, do I understand it correct that the issues, which are described for cache mode off can only occur if I would edit on the NAS a file located on the remote?
https://rclone.org/commands/rclone_serve_webdav/

Then it should be no issue to deactivate the cache for my use case.

If cloud sync fails to upload a file it would retry it anyway?

4

some apps need the cache, some apps do not.
if you do not want to use the cache for some reason, have to do testing.
and that testing will not be trivial to do.

by default, once a file is in the rclone vfs cache, rclone waits just five seconds before uploading it to pcloud.

if you are worried that cloudsync is continuous sync, i would not worry.
let's say there is a file named file.txt, you edit that file, so now cloudsync will upload file.txt into the rclone cache.
after five seconds, rclone will start to upload file.txt from its cache to pcloud.
now at the same time rclone is uploading file.txt from its cache, you edit file.txt again.
cloudsync notices that change, uploads it to the rclone cache.
rclone will stop the current upload of the older file.txt to pcloud, and start to upload the newer file.txt

5

I think my concern is primary related for uploads of big amount of data. Should occur only once.

So at the moment I initially upload the NAS content to the cloud, which are some dozen GB. In this case Cloudsync "copy" everthing to the cache and show "everything is synced". But that is wrong, because its only copied to the cache and the upload will take some days... (And I cannot limit this uploads to the night, when it is not limited by cloud sync and I do not shut down the NAS during daytime).

Afterwards changes are not so big regarding data size.

But I could try what happens if the cache is deactivated and I change a file which is curently uploaded.

6

instead of cloudsync + rclone serve webdav, do what i do.
just run rclone copy from local to pcloud.

with rclone, you can change the upload bandwidth on the fly.
using a schedule a simple script, you can slow down rclone for a period of time, then speed it up
--bwlimit

7

ok. thank you.

For inital upload copy would be an option, yes. Afterwards I want to have a (bidirectional) sync.

Is it possible that rclone prevent the diskstation from standby? But perhaps there is also another reason for that.

8

i have never used standby with a synbox, but i would assume that if an application is using the hard drive, then synbox cannot go into standby.
all synboxes has UPS.

9

Yes, of course. During activity there will be no standby.

I made a trial:

  • deactivated cache
  • created locally a new file and opened it for edting on my computer
  • cloud sync uploaded the file without error (file was still open)
  • I changed the file and saved
  • cloud sync uploaded the changed file

So seems to work for such a use case.

10

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.