Sync one source to multiple host

What is the problem you are having with rclone?

When mount a Amazon S3 Compliant Storage (Tencent COS) on debian 11.1 in more than one vps machines ( all vps and the COS are in the some intranet ), then touch a file in one vps, it takes about 3-5 minutes for the file created to display in other vps(e.g using ls command ). the same as the rm command for minutes to take effect in other host. So is this normal, or some cache reason, is it possible for the operation to take effect in time?

Run the command 'rclone version' and share the full output of the command.

rclone v1.53.3-DEV
- os/arch: linux/amd64
- go version: go1.15.9

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads

I tried rclone version 1.66.0 and 1.53.3 (the default version in apt repository) they act the same.

Which cloud storage system are you using? (eg Google Drive)

Tencent COS

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount cos:/cos-4-5-1319047051  /www/wp-content/uploads --cache-dir /temp --vfs-cache-mode writes  --uid=65534 --gid=65534 --no-modtime --daemon

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[cos]
type = s3
provider = TencentCOS
access_key_id = XXX
secret_access_key = XXX
endpoint = cos.ap-shanghai.myqcloud.com
storage_class = STANDARD
acl = defaul

A log from the command that you were trying to run with the -vv flag

2024/05/11 16:24:30 DEBUG : rclone: Version "v1.66.0" starting with parameters ["rclone" "mount" "cos:/cos-4-5-1319047051" "/www/wp-content/uploads" "--cache-dir" "/temp" "--vfs-cache-mode" "writes" "--uid=65534" "--gid=65534" "--no-modtime" "-vv"]
2024/05/11 16:24:30 DEBUG : Creating backend with remote "cos:/cos-4-5-1319047051"
2024/05/11 16:24:30 DEBUG : Using config file from "/root/.config/rclone/rclone.conf"
2024/05/11 16:24:30 DEBUG : Resolving service "s3" region "us-east-1"
2024/05/11 16:24:30 DEBUG : fs cache: renaming cache item "cos:/cos-4-5-1319047051" to be canonical "cos:cos-4-5-1319047051"
2024/05/11 16:24:30 INFO  : S3 bucket cos-4-5-1319047051: poll-interval is not supported by this remote
2024/05/11 16:24:30 DEBUG : vfs cache: root is "/temp"
2024/05/11 16:24:30 DEBUG : vfs cache: data root is "/temp/vfs/cos/cos-4-5-1319047051"
2024/05/11 16:24:30 DEBUG : vfs cache: metadata root is "/temp/vfsMeta/cos/cos-4-5-1319047051"
2024/05/11 16:24:30 DEBUG : Creating backend with remote "/temp/vfs/cos/cos-4-5-1319047051"
2024/05/11 16:24:30 DEBUG : Creating backend with remote "/temp/vfsMeta/cos/cos-4-5-1319047051"
2024/05/11 16:24:30 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item hehe not removed, freed 0 bytes
2024/05/11 16:24:30 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item iii not removed, freed 0 bytes
2024/05/11 16:24:30 INFO  : vfs cache: cleaned: objects 2 (was 2) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2024/05/11 16:24:30 DEBUG : S3 bucket cos-4-5-1319047051: Mounting on "/www/wp-content/uploads"
2024/05/11 16:24:30 DEBUG : : Root: 
2024/05/11 16:24:30 DEBUG : : >Root: node=/, err=<nil>
2024/05/11 16:25:21 DEBUG : /: Attr: 
2024/05/11 16:25:21 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2024/05/11 16:25:22 DEBUG : /: Attr: 
2024/05/11 16:25:22 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2024/05/11 16:25:22 DEBUG : /: ReadDirAll: 
2024/05/11 16:25:22 DEBUG : /: >ReadDirAll: item=3, err=<nil>
2024/05/11 16:25:22 DEBUG : /: Lookup: name="iii"
2024/05/11 16:25:22 DEBUG : /: >Lookup: node=iii, err=<nil>
2024/05/11 16:25:22 DEBUG : iii: Attr: 
2024/05/11 16:25:22 DEBUG : iii: >Attr: a=valid=1s ino=0 size=0 mode=-rw-r--r--, err=<nil>
2024/05/11 16:25:27 DEBUG : /: Attr: 
2024/05/11 16:25:27 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>
2024/05/11 16:25:27 DEBUG : /: ReadDirAll: 
2024/05/11 16:25:27 DEBUG : /: >ReadDirAll: item=3, err=<nil>
2024/05/11 16:25:30 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item hehe not removed, freed 0 bytes
2024/05/11 16:25:30 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item iii not removed, freed 0 bytes
2024/05/11 16:25:30 INFO  : vfs cache: cleaned: objects 2 (was 2) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2024/05/11 16:26:30 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item hehe not removed, freed 0 bytes
2024/05/11 16:26:30 DEBUG : vfs cache RemoveNotInUse (maxAge=3600000000000, emptyOnly=false): item iii not removed, freed 0 bytes
2024/05/11 16:26:30 INFO  : vfs cache: cleaned: objects 2 (was 2) in use 0, to upload 0, uploading 0, total size 0 (was 0)

Yes it is normal for non polling remotes.

Also you are using ancient rclone version. Many years old one.

welcome to the forum,

the default value is 5 minutes, but that can be reduced by using --dir-cache-time
tho, that will consume more API calls and possible additional costs from the provider.
for example, wasabi and idrive do not charge for api calls, whereas, aws does charge for api calls.

rclone has two caches, vfs dir cache and vfs file cache.
can check out my summary of the rclone caches

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.