What is the problem you are having with rclone?
I am running object storage on top of Google Drive, which generates a massive number of 4MB silce files uploads. I am looking for ways to increase the number of upload threads for Google Drive, as the current speed of uploading 2 to 3 files per second with a single client cannot keep up with the rate at which my object storage generates new files.
I have tried sharing folders with a Google Group (containing 90 service accounts), but because files might not be uploaded by the corresponding SA (uploaded by the main account or other SAs), this leads to a problem where even though rclone can use SAs, some files cannot be deleted normally.
Is there a way to use service accounts to increase the actual upload thread count in rclone mount(to personal drive)?
Run the command 'rclone version' and share the full output of the command.
rclone v1.69.0
- os/version: debian 11.11 (64 bit)
- os/kernel: 6.12.6-x64v3-xanmod1 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.23.4
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
personal drive in google workspace(non workspace admin)(so it is Google Drive)
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone mount my_main_drive:/redact /var/redact --default-permissions --allow-non-empty --allow-other --transfers 32 --checkers 32 --low-level-retries 5 --copy-links --umask 0000 --log-level INFO --buffer-size 0M --vfs-cache-mode writes --vfs-fast-fingerprint --vfs-cache-max-age 1m --vfs-cache-max-size 2048M --vfs-write-back 16s --rc --rc-enable-metrics --rc-no-auth --rc-addr 127.0.0.1:5572
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[my_main_drive]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive =
server_side_across_configs = true
A log from the command that you were trying to run with the -vv flag
since the rclone work as intent,so there's no logs(about error or something)