What is the problem you are having with rclone?
I'm using S3 as a mount for rclone on my Windows system. Specifically, I am using it for Steam games. However, when a new game is installed, it takes a very long time, as it does a sequential server-side copy one file at a time from one folder in the S3 bucket to another. You can see this in the log where only a single file is being copied from 19:49:29 to 19:50:44, and all other processes stop.
Is it possible to continue doing other server-side copies in parallel (or to disable server-side copy entirely)? If so what do I need to do to change my command?
Run the command 'rclone version' and share the full output of the command.
rclone v1.64.0
- os/version: Microsoft Windows Server 2022 Datacenter 21H2 (64 bit)
- os/kernel: 10.0.20348.587 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.21.1
- go/linking: static
- go/tags: cmount
Which cloud storage system are you using? (eg Google Drive)
AWS S3
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone mount "gameaway-steam-games:gameaway-steam-games/!email!" "C:\Program Files (x86)\Steam\steamapps" --allow-other --vfs-cache-mode full --vfs-cache-max-age 0s --fuse-flag --FileSecurity="D:P(A;;FA;;;WD)" --transfers 1000 --fast-list --buffer-size 1G --checkers 32 --vfs-read-chunk-size 16M --vfs-read-chunk-size-limit off --bwlimit 100G --vfs-read-ahead 128M --no-checksum --no-modtime -v --log-file="C:\Startup Scripts\rclone.log"
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[gameaway-steam-games]
type = s3
provider = AWS
env_auth = true
region = ap-south-1
location_constraint = ap-south-1
### Double check the config for sensitive info before posting publicly
A log from the command that you were trying to run with the -vv flag
2023/09/17 19:49:28 INFO : downloading/1466860/cardinal/archives/Gen7AssetsGeometryLODs1.sga: Copied (server-side copy) to: common/Age of Empires IV/cardinal/archives/Gen7AssetsGeometryLODs1.sga
2023/09/17 19:49:28 INFO : downloading/1466860/cardinal/archives/Gen7AssetsGeometryLODs1.sga: Deleted
2023/09/17 19:49:29 INFO : downloading/1466860/cardinal/archives/Gen7AssetsGeometryLODs2.sga: Copied (server-side copy) to: common/Age of Empires IV/cardinal/archives/Gen7AssetsGeometryLODs2.sga
2023/09/17 19:49:29 INFO : downloading/1466860/cardinal/archives/Gen7AssetsGeometryLODs2.sga: Deleted
2023/09/17 19:49:29 INFO : downloading/1466860/cardinal/archives/Gen7AssetsGeometryLODs3.sga: Copied (server-side copy) to: common/Age of Empires IV/cardinal/archives/Gen7AssetsGeometryLODs3.sga
2023/09/17 19:49:29 INFO : downloading/1466860/cardinal/archives/Gen7AssetsGeometryLODs3.sga: Deleted
2023/09/17 19:50:44 INFO : downloading/1466860/cardinal/archives/Gen7AssetsMaterials.sga: Copied (server-side copy) to: common/Age of Empires IV/cardinal/archives/Gen7AssetsMaterials.sga
2023/09/17 19:50:44 INFO : downloading/1466860/cardinal/archives/Gen7AssetsMaterials.sga: Deleted
2023/09/17 19:50:45 INFO : downloading/1466860/cardinal/archives/Gen7ShadersAndEDFs.sga: Copied (server-side copy) to: common/Age of Empires IV/cardinal/archives/Gen7ShadersAndEDFs.sga
2023/09/17 19:50:45 INFO : downloading/1466860/cardinal/archives/Gen7ShadersAndEDFs.sga: Deleted