VFS Cache Uploading Waits Until Entire File in Cache, Issue with --bwlimit

What is the problem you are having with rclone?

I am uploading files to google drive and when I use --vfs-cache-mode full or --vfs-cache-mode writes the file I am uploading will copy to the cache location at the rate of --bwlimit and the upload won't start until the entire file is in the cache. Once the file is complete in the cache it is uploaded at my maximum internet speed, ignoring --bwlimit while the next file will copy to the cache once again at the --bwlimit.
In other words, my goal is to have the file copied to cache at maximum drive speed and then uploaded using --bwlimit or I would even be happy if both the file copied at --bwlimit to cache and upload occurs at --bwlimit simultaneously rather than waiting for the file to be copied completely to the cache first.

What is your rclone version (output from rclone version)

v1.56.0

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows 10 64 bit

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy --size-only --retries=1 --progress --fast-list --bwlimit=8.5M --log-file "C:\Program Files\rclone\logs\log.txt" "E:\Subs" "A:\Subs"

The rclone config contents with secrets removed.

[gdrive]
type = drive
client_id = 
client_secret = 
scope = drive
token = 
team_drive = 

[gcache]
type = cache
remote = gdrive:/gdrive
plex_url = url:port
plex_username = 
plex_password = 
info_age = 1d
plex_token = 

[gcrypt2]
type = crypt
remote = gcache:/crypt
filename_encryption = obfuscate
password = 
password2 = 

A log from the command with the -vv flag

2021/08/30 00:43:50 INFO  : Starting bandwidth limiter at 8.500Mi Byte/s
2021/08/30 00:43:50 DEBUG : rclone: Version "v1.56.0" starting with parameters ["rclone" "copy" "--size-only" "--retries=1" "--progress" "--fast-list" "--bwlimit=8.5M" "--log-file" "C:\\Program Files\\rclone\\logs\\log.txt" "E:\\Subs" "A:\\Subs" "--transfers=1" "-vv"]
2021/08/30 00:43:50 DEBUG : Creating backend with remote "E:\\Subs"
2021/08/30 00:43:50 DEBUG : Using config file from \\AppData\\Roaming\\rclone\\rclone.conf"
2021/08/30 00:43:50 DEBUG : fs cache: renaming cache item "E:\\Subs" to be canonical "//?/E:/Subs"
2021/08/30 00:43:50 DEBUG : Creating backend with remote "A:\\Subs"
2021/08/30 00:43:50 DEBUG : fs cache: renaming cache item "A:\\Subs" to be canonical "//?/A:/Subs"

Here is my mount as well:

mount --config C:\Users\Name\AppData\Roaming\rclone\rclone.conf --cache-dir B:\rcloneCache --dir-cache-time=8760h --attr-timeout=8700h --drive-chunk-size=64M --vfs-cache-mode=full --vfs-cache-max-size 200G --buffer-size=256M --vfs-read-ahead 2G --vfs-cache-poll-interval=15s --poll-interval=15s gcrypt2: A:

Files can only be uploaded once they are fully copied as there is no concept of partial files.

It will copy the whole file to the cache and then upload.

I don't see any upload in the log though so there isn't much to troubleshoot.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.