STOP and READ USE THIS TEMPLATE NO EXCEPTIONS - By not using this, you waste your time, our time and really hate puppies. Please remove these two lines and that will confirm you have read them.
What is the problem you are having with rclone?
I did a cross account mounting of S3 bucket in a g4dn.4xlarge EC2 instance. When I try to ingest any object (~260 GB) to the 3rd party simulation tool, the playback speed is very less. To debug it more when I try to copy any object (~260 GB) from the mounted drive to any location in the attached EBS volume(IOPS = 4000 and ThroughPut = 1000 MiB/S)max copy speed is approx. 50 MB/s
So wanted to know which other parameters I can use to mount the bucket so that I can ingest the file directly to the simulation tool.
PS:
- I cannot use rclone copy or copy via aws cli to have a offline copy as in my use-case, I need to ingest the objects(> ~260 GB) to a simulation tool.
Run the command 'rclone version' and share the full output of the command.
rclone v1.70.3
- os/version: Microsoft Windows Server 2022 Datacenter 21H2 21H2 (64 bit)
- os/kernel: 10.0.20348.3932 (x86_64)
- os/type: windows
- os/arch: amd64
- go/version: go1.24.4
- go/linking: static
- go/tags: cmount
Which cloud storage system are you using? (eg Google Drive)
AWS S3
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone --config "C:\utils\rclone\rclone_aws_s3_eu-west-1_env_auth.conf" mount aws_s3:/my-test-bucket X:\ --vfs-cache-mode minimal --dir-cache-time 1m --poll-interval 15s --ignore-checksum --verbose
Measure-Command {Copy-Item "X:\20110101033910\Test.rrec" "D:\TestData" }/
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[aws_s3]
type = s3
provider = AWS
env_auth = true
region = eu-west-1
acl = bucket-owner-full-control
location_constraint = eu-west-1
A log from the command that you were trying to run with the -vv flag
Logs attached - Rclone.log
Rclone.log (3.3 MB)