What is the problem you are having with rclone?
How does rclone's VFS cache work? I'm running an object storage on top of rclone mount (which generates a large number of 4MB small files), and since the remote storage is Google Drive (with a limit of uploading only 2-3 files per second per client), rclone's file upload speed can't keep up with the rate at which the object storage generates new files. Can I use autorclone with multiple SA accounts to first upload the files in the VFS cache directory to a team drive, and then move them to the actual remote directory (located on a personal drive)? If possible, how should I do this? Should I copy the files from the VFS directory or the vfsmeta directory? After I've uploaded the files myself, how can I let rclone know that I've already uploaded them to avoid duplicate uploads?
Run the command 'rclone version' and share the full output of the command.
rclone v1.69.0
- os/version: debian 11.11 (64 bit)
- os/kernel: 6.12.6-x64v3-xanmod1 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.4
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
personal drive in google workspace(non workspace admin)(so it is Google Drive),some team drive also included
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone mount my_main_drive:/redact /var/redact --default-permissions --allow-non-empty --allow-other --transfers 32 --checkers 32 --low-level-retries 5 --copy-links --umask 0000 --log-level INFO --buffer-size 0M --vfs-cache-mode writes --vfs-fast-fingerprint --vfs-cache-max-age 1m --vfs-cache-max-size 2048M --vfs-write-back 16s --rc --rc-enable-metrics --rc-no-auth --rc-addr 127.0.0.1:5572
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[my_main_drive]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive =
server_side_across_configs = true
A log from the command that you were trying to run with the -vv flag