Call vfs/refresh API friendly

Help and Support
1

What is the problem you are having with rclone?

I have a combine remote that contains 15 SharePoint remotes. I use it with a mount service.

Every night I want to trigger a vfs/refresh, which starts a huge directory-listing task across all remotes inside the combined remote. This part works as intended.

However, it causes pacer slowdowns because of SharePoint rate limits, which makes the remote “force-sleep” for a while.
Because of these rate limits, I do not want vfs/refresh to run automatically when the mount service starts.

My question is:

Is there a way to perform a “soft refresh” with rc by using --tpslimit or --tpslimit-burst 0? Maybe with --bwlimit?

Are there any other options to reduce the rate-limit hits?

Are global options in an rc call respected by a mount? or are the options privileged set by the service?

I want to keep the tpslimit for vfs/refresh lower as for the mount, since the mount is not under heavy load.

All 15 remotes use their own client ID (app ID).

Run the command 'rclone version' and share the full output of the command.

rclone v1.73.0-beta.9354.e5e6a4b5a

  • os/version: ubuntu 24.04 (64 bit)
  • os/kernel: 6.8.0-87-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.25.5
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Microsoft bussiness / Sharepoint

The command you were trying to run:

rclone rc vfs/refresh recursive=true --url 127.0.0.1:5572 _async=true --tpslimit 0.5 --tpslimit-burst 0 --bwlimit 5M

'rclone config redacted'

[together]
type = combine
upstreams = julian_team_privat=julian_team_privat: testteam2_privat=testteam2_privat: testteam3_privat=testteam3_privat: testteam4_privat=testteam4_privat: testteam5_privat=testteam5_privat: team6_privat=team6_privat: team7_privat=team7_privat: team8_privat=team8_privat: team9_privat=team9_privat: team10_privat=team10_privat: team11_privat=team11_privat: team12_privat=team12_privat: team13_privat=team13_privat: team14_privat=team14_privat: team15_privat=team15_privat:

[testteam5_privat]
type = crypt
remote = testteam5:privat
password = XXX
password2 = XXX
server_side_across_configs = true

[team9_privat]
type = crypt
remote = testteam5:privat
password = XXX
password2 = XXX
server_side_across_configs = true

etc....

mount service


[Unit]
Description=rclone: Remote %i
Documentation=man:rclone(1)
After=network-online.target
Wants=network-online.target 
AssertPathIsDirectory=%h/mnt/lowapi
StartLimitInterval=200
StartLimitBurst=5

[Service]
Type=notify
ExecStart= \
  /usr/bin/rclone mount \
    --config=%h/.config/rclone/rclone.conf \
    --log-level DEBUG \
    --log-file /root/logs/rclone/lowapi.log \
    --log-file-max-size 2G \
    --log-file-max-backups 1 \
    --umask 022 \
--vfs-cache-mode full \
--allow-other \
--bind 0.0.0.0 \
--no-modtime \
--cache-dir /root/rclone/cache/lowapi \
--server-side-across-configs \
--onedrive-av-override \
--use-server-modtime \
--no-checksum \
--rc-addr=127.0.0.1:5572 \
--rc \
--rc-web-gui-no-open-browser \
--rc-no-auth \
--rc-web-gui \
--disable-http2 \
--vfs-fast-fingerprint \
--attr-timeout 8700h \
--ignore-checksum \
--no-check-certificate \
--onedrive-hard-delete \
--no-update-modtime \
--checkers 1 \
--tpslimit 0.5 \
--transfers 1 \
--bwlimit 13M:8M \
--low-level-retries 10 \
--onedrive-no-versions \
--onedrive-hash-type none \
--onedrive-delta \
--onedrive-chunk-size 250M \
--dir-cache-time 999999h \
--vfs-cache-max-age 9999h \
--buffer-size 32M \
--vfs-write-back 15m \
--vfs-read-chunk-size 32M \
--vfs-read-chunk-size-limit off \
--vfs-read-ahead 64M \
--vfs-read-chunk-streams 0 \
--poll-interval 0 \
--ignore-size \
--user-agent "NONISV|7a-09187b47aa3a" \
--vfs-cache-max-size 20G \
    %i: %h/mnt/lowapi
ExecStop=/bin/fusermount -u %h/mnt/lowapi
Restart=always
RestartSec=30

[Install]
WantedBy=default.target

A log from the command that you were trying to run with the -vv flag

2025/12/11 11:53:33 DEBUG : rc: "vfs/refresh": with parameters map[_async:true recursive:true]
2025/12/11 11:53:33 DEBUG : rc: "vfs/refresh": reply map[executeId:b4cdfbd3-d32f-4b1a-8f8f-58cec3b4e26d jobid:7]: <nil>
2025/12/11 11:53:33 DEBUG : Reading directory tree

2025/12/11 11:55:35 DEBUG : Too many requests. Trying again in 5 seconds.
2025/12/11 11:55:35 DEBUG : pacer: low level retry 1/10 (error activityLimitReached: throttledRequest: The request has been throttled: trying again in 5s)
2025/12/11 11:55:35 DEBUG : pacer: Rate limited, increasing sleep to 5s
2025/12/11 11:55:36 DEBUG : pacer: Reducing sleep to 2.8125s
2025/12/11 11:55:36 DEBUG : rc: "core/stats": with parameters map[]
2025/12/11 11:55:36 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletedDirs:1972 deletes:0 elapsedTime:248.544159976 errors:0 eta:<nil> fatalError:false listed:58650 renames:0 retryError:false serverSideCo
pies:0 serverSideCopyBytes:0 serverSideMoveBytes:0 serverSideMoves:0 speed:0 totalBytes:0 totalChecks:0 totalTransfers:0 transferTime:0 transfers:0]: <nil>
2025/12/11 11:55:36 DEBUG : rc: "core/stats": with parameters map[]
2025/12/11 11:55:36 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletedDirs:1972 deletes:0 elapsedTime:248.547638101 errors:0 eta:<nil> fatalError:false listed:58650 renames:0 retryError:false serverSideCo
pies:0 serverSideCopyBytes:0 serverSideMoveBytes:0 serverSideMoves:0 speed:0 totalBytes:0 totalChecks:0 totalTransfers:0 transferTime:0 transfers:0]: <nil>
2025/12/11 11:55:37 DEBUG : Too many requests. Trying again in 5 seconds.
2025/12/11 11:55:37 DEBUG : pacer: low level retry 1/10 (error activityLimitReached: throttledRequest: The request has been throttled: trying again in 5s)
2025/12/11 11:55:37 DEBUG : pacer: Rate limited, increasing sleep to 5s
2025/12/11 11:55:41 DEBUG : rc: "core/stats": with parameters map[]
2025/12/11 11:55:41 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletedDirs:1972 deletes:0 elapsedTime:253.532513991 errors:0 eta:<nil> fatalError:false listed:58650 renames:0 retryError:false serverSideCo
pies:0 serverSideCopyBytes:0 serverSideMoveBytes:0 serverSideMoves:0 speed:0 totalBytes:0 totalChecks:0 totalTransfers:0 transferTime:0 transfers:0]: <nil>
2025/12/11 11:55:41 DEBUG : rc: "core/stats": with parameters map[]
2025/12/11 11:55:41 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletedDirs:1972 deletes:0 elapsedTime:253.532674774 errors:0 eta:<nil> fatalError:false listed:58650 renames:0 retryError:false serverSideCo
pies:0 serverSideCopyBytes:0 serverSideMoveBytes:0 serverSideMoves:0 speed:0 totalBytes:0 totalChecks:0 totalTransfers:0 transferTime:0 transfers:0]: <nil>
2025/12/11 11:55:41 DEBUG : Too many requests. Trying again in 5 seconds.
2025/12/11 11:55:41 DEBUG : pacer: low level retry 2/10 (error activityLimitReached: throttledRequest: The request has been throttled: trying again in 5s)
2025/12/11 11:55:46 DEBUG : rc: "core/stats": with parameters map[]
2025/12/11 11:55:46 DEBUG : rc: "core/stats": reply map[bytes:0 checks:0 deletedDirs:1972 deletes:0 elapsedTime:258.548642576 errors:0 eta:<nil> fatalError:false listed:60650 renames:0 retryError:false serverSideCo
pies:0 serverSideCopyBytes:0 serverSideMoveBytes:0 serverSideMoves:0 speed:0 totalBytes:0 totalChecks:0 totalTransfers:0 transferTime:0 transfers:0]: <nil>
2025/12/11 11:55:46 DEBUG : rc: "core/stats": with parameters map[]

Thank you in advance

2

Try removing –onedrive-delta. I’ve been having problems this week with Delta and pacing this month. I removed it and running the old school (slower) refresh has been working fine. Not sure if Microsoft has a problem with the API or something else.

Consider that a refresh for the 15 drives might take too long in particular with a tpslimit of 0.5. Based on my quick math, a tps of 2 should be compatible with Microsoft API quotas.

You could also avoid the whole refresh by enabling poll-interval as it’s supported by OneDrive and that’s the whole reason of delta, to provide the delta changes on the file structure. Depending on your use case it can even be somewhat long enough (around 5 minutes) if you do all the changes directly on your mount.

1 Like
3

Try removing –onedrive-delta. I’ve been having problems this week with Delta and pacing this month. I removed it and running the old school (slower) refresh has been working fine. Not sure if Microsoft has a problem with the API or something else.

Interesting, I also had problems this month…
In fact: i’m running the combine twice. One for streaming with tpslimit 2 and one for downloading.

Consider that a refresh for the 15 drives might take too long in particular with a tpslimit of 0.5. Based on my quick math, a tps of 2 should be compatible with Microsoft API quotas.

in the night with tpslimit 2 it finished within 5 Minutes. shure 0.5 longer but thats not a big deal.

You could also avoid the whole refresh by enabling poll-interval as it’s supported by OneDrive and that’s the whole reason of delta, to provide the delta changes on the file structure. Depending on your use case it can even be somewhat long enough (around 5 minutes) if you do all the changes directly on your mount.

Hmm… I’m wondering:

if I set a poll interval, for example --poll-interval 24h, do I need to set --dir-cache-time to something lower? Wouldn’t that make the API usage more aggressive?

Emby is running on the second combined mount (with tpslimit=2). Because of that, I would prefer the folders to update only every 24 hours, which is enough since I run a full library scan every night.

In short: I would like the mount to refresh only once every 24 hours.

4

I’ve set –-poll-interval 60m but folder wont show up I’ve created on mount A when I create them in mount B…. maybe because of the high dir-cache-time?

5

Ok,

Based on what I saw while watching the debug logs:

  • --dir-cache-time can be set to a high value if needed.

  • --poll-interval clears the directory cache in its requesting sector as soon as changes are detected there. I’m testing 30 minutes now to see how it works.

I’ll keep using --onedrive-delta because I only refresh once when the mount starts. After that, updates will be handled by the poll interval.

For reference:
I split one combined remote into two separate mount services with different TPS limits. OneDrive is very strict with API usage, so I need to manage resources carefully.

  • The first mount is for user experience (Emby) and uses tpslimit 2 with a burst of 3.

  • The second mount is for Radarr, Sonarr, etc. and uses tpslimit 1, since these applications don’t need fast or reactive updates. Even for empty dirs!

  • Changes to tpslimit, dir-cache-time or poll-interval cannot be altered with rc command. The mount needs a full restart for this. The only “on-the-fly change” is bandwith-limit.

  • I’ll throttle bandwidth in the night for scanning libraries with emby.

Thanks for the feedback @jcastill . I’ll test this setup further. If anything fails, I’ll rethink it — but for now, the setup is very stable, even with Microsoft. :slight_smile:

My final services for the mounts

Emby mount


# User service for Rclone mounting
#
# Place in ~/.config/systemd/user/
# File must include the '@' (ex rclone@.service)
# As your normal user, run 
#   systemctl --user daemon-reload
# You can now start/enable each remote by using rclone@<remote>..
#   systemctl --user enable rclone@dropbox
#   systemctl --user start rclone@dropbox

[Unit]
Description=rclone: Remote FUSE filesystem for cloud storage config %i
Documentation=man:rclone(1)
After=network-online.target
Wants=network-online.target 
AssertPathIsDirectory=%h/mnt/%i
StartLimitInterval=200
StartLimitBurst=5

[Service]
Type=notify
Environment="passwd"
ExecStart= \
  /usr/bin/rclone mount \
    --config=%h/.config/rclone/rclone.conf \
    --log-level DEBUG \
    --log-file /root/logs/rclone/%i.log \
    --log-file-max-size 2G \
    --log-file-max-backups 1 \
    --umask 022 \
--vfs-cache-mode full \
--allow-other \
--bind 0.0.0.0 \
--no-modtime \
--read-only \
--cache-dir /root/rclone/cache \
--server-side-across-configs \
--onedrive-av-override \
--use-server-modtime \
--no-checksum \
--rc-addr=127.0.0.1:5571 \
--rc-web-gui \
--rc-web-gui-no-open-browser \
--rc \
--rc-no-auth \
--disable-http2 \
--vfs-fast-fingerprint \
--ignore-checksum \
--no-check-certificate \
--onedrive-hard-delete \
--no-update-modtime \
--checkers 1 \
--low-level-retries 10 \
--vfs-write-back 60s \
--onedrive-no-versions \
--onedrive-hash-type none \
--onedrive-delta \
--onedrive-chunk-size 150M \
--dir-cache-time 999999h \
--exclude ".DS_Store" \
--exclude "Thumbs.db" \
--exclude "@eaDir" \
--exclude "._*" \
--exclude "*.{part*,ytdl,tmp,temp,crdownload,download,aria*,bak}" \
--vfs-refresh \
--attr-timeout 8700h \
--vfs-cache-max-age 720h \
--tpslimit 1.8 \
--tpslimit-burst 3 \
--transfers 1 \
--buffer-size 128M \
--vfs-read-chunk-size 32M \
--vfs-read-chunk-size-limit off \
--vfs-read-ahead 128M \
--vfs-read-chunk-streams 0 \
--poll-interval 30m \
--ignore-size \
--user-agent "NONISV|" \
--vfs-cache-max-size 80G \
    %i: %h/mnt/%i
ExecStop=/bin/fusermount -u %h/mnt/%i
Restart=always
RestartSec=30

[Install]
WantedBy=default.target

lowapi mount

# User service for Rclone mounting
#
# Place in ~/.config/systemd/user/
# File must include the '@' (ex rclone@.service)
# As your normal user, run 
#   systemctl --user daemon-reload
# You can now start/enable each remote by using rclone@<remote>..
#   systemctl --user enable rclone@dropbox
#   systemctl --user start rclone@dropbox

[Unit]
Description=rclone: Remote %i
Documentation=man:rclone(1)
After=network-online.target
Wants=network-online.target 
AssertPathIsDirectory=%h/mnt/lowapi
StartLimitInterval=200
StartLimitBurst=5

[Service]
Type=notify
Environment="password"
ExecStart= \
  /usr/bin/rclone mount \
    --config=%h/.config/rclone/rclone.conf \
    --log-level DEBUG \
    --log-file /root/logs/rclone/lowapi.log \
    --log-file-max-size 2G \
    --log-file-max-backups 1 \
    --umask 022 \
--vfs-cache-mode full \
--allow-other \
--bind 0.0.0.0 \
--no-modtime \
--cache-dir /root/rclone/cache/lowapi \
--server-side-across-configs \
--onedrive-av-override \
--use-server-modtime \
--no-checksum \
--rc-addr=127.0.0.1:5572 \
--rc \
--rc-web-gui-no-open-browser \
--rc-no-auth \
--rc-web-gui \
--disable-http2 \
--vfs-fast-fingerprint \
--attr-timeout 8700h \
--ignore-checksum \
--no-check-certificate \
--onedrive-hard-delete \
--no-update-modtime \
--checkers 1 \
--tpslimit 1 \
--tpslimit-burst 0 \
--transfers 1 \
--bwlimit 20M \
--low-level-retries 10 \
--onedrive-no-versions \
--onedrive-hash-type none \
--onedrive-delta \
--onedrive-chunk-size 250M \
--dir-cache-time 999999h \
--vfs-cache-max-age 9999h \
--buffer-size 32M \
--vfs-write-back 10m \
--vfs-read-chunk-size 32M \
--vfs-read-chunk-size-limit off \
--vfs-read-ahead 64M \
--vfs-read-chunk-streams 0 \
--poll-interval 30m \
--ignore-size \
--user-agent "NONISV|a3ad" \
--vfs-cache-max-size 20G \
    %i: %h/mnt/lowapi
ExecStop=/bin/fusermount -u %h/mnt/lowapi
Restart=always
RestartSec=30

[Install]
WantedBy=default.target
1 Like