Rclone - specify path entry point on s3 for syncing

Hi,

I'm trying to do regular syncs of a public Amazon s3 bucket to local storage. In this case, it is a software package repository (.deb files) that I would like to mirror locally to allow access to systems behind a firewall. The issue I'm having is that I can't find a way to specify a remote path to sync from in s3. The only thing I can do is tell it which files to include or exclude, so the end result is that it includes the base directories which I don't want in my local mirror. The remote repository used to be on standard non-bucketed web hosting which allows specifying the remote path in the URL, but I can't see a way to do this in s3. Does anyone know if this is possible? I can't restructure the local paths to accommodate the base directories, and the option of syncing with the base directories included then doing a selective sync locally a second time is not feasible either.

rclone v1.47.0, CentOS 7.6 64-bit

The command I'm running:

RCLONE_CONFIG_S3_TYPE=s3 RCLONE_CONFIG_S3_PROVIDER=Other RCLONE_CONFIG_S3_ENV_AUTH=false RCLONE_CONFIG_S3_ENDPOINT=https://s3.repo.saltstack.com rclone sync --fast-list --use-server-modtime --include 'py3/ubuntu/**' -v s3:s3/ ./ubuntu

So what I end up with locally is: './ubuntu/py3/ubuntu/', when I just want to sync all the stuff under py3/ubuntu directly to ./ubuntu/

Any help much appreciated!

-Cam

Put your remote path on here

s3:s3/path/goes/here

hi,
latest rclone is v1.53.1

you can update using this

https://rclone.org/downloads/#script-download-and-install

Thanks for the reply Craig, that indeed works fine. I meant to try that before posting but for some reason did not.

Cheers,

Cam

1 Like