What is the problem you are having with rclone?
We are using rclone to backup our SQL Server databases to an AWS S3 bucket. We are able to backup the databases without any issues. What we found was that when we use the cache option with rclone, rclone copies the backups to the cache first and then moves it to the S3 bucket. That means that our cache on the local disk has to be the size of all the databases backups combined which runs into GB's for a single days worth of backups. We have a database backup that is nearly 400GB in size and that would force us to have a cache on the local storage for that equal amount of space. The reason we are migrating to AWS is to reduce storage costs. But the Cache storage will still require us to provide that to rclone. Is there a way to perhaps reduce the no of files that are cached to reduce the cache space? What are any best practises that you can recommend in this scenario. Fyi. all the backups are sequential.
Thank you very much for your time and help.
What is your rclone version (output from rclone version
)
< V 1.52.3
-->
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Microsoft Windows 2013, 64 bit
Which cloud storage system are you using? (eg Google Drive)
AWS
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
Paste command here
``` start E:\rclone\rclone mount test:crico-sql-prod z: --vfs-cache-mode writes --allow-other --config=c:\temp\rclone\rclone.conf --log-file e:\rclone.log --log-level DEBUG
#### The rclone config contents with secrets removed.
<!-- You should use 3 backticks to begin and end your paste to make it readable. -->
Paste config here
type = s3
provider = AWS
env_auth = false
access_key_id = ####################
secret_access_key = ##########################################
region = us-east-1
acl = private
storage_class = STANDARD
#### A log from the command with the `-vv` flag
<!-- You should use 3 backticks to begin and end your paste to make it readable. Or use a service such as https://pastebin.com or https://gist.github.com/ -->
Paste log here