S3 to SMB data copy/sync with metadata mtime key update in destination

Help and Support
1

What is the problem you are having with rclone?

Hello, I'm trying to download and synchronize data from S3 to SMB share. The files are being uploaded and regularly synchronized with AWS DataSync from Windows based file servers (SMB) to this S3 bucket, so effectivelly we're doing SMB to S3 with AWS DataSync and then S3 to SMB with rclone.
I'm not having any issue per se with rsync downloading / synchroniznig the data from S3, however I'm wondering if there's an option that would allow rsync to read a user-defined meta-data key "file-mtime" on the file objects in S3 (it's the original file time stamps from the source servers) and write the data down to our SMB share while also updating the time stamps on files in destination based on this meta-data key.
I guess my primary point is that say a file in the original file server has a timestamp of 1/1/1999, when it gets uploaded to S3, the file object gets a timestamp of the upload date/time, e.g. 9/11/2025. The original date is stored in the user-defined meta-data key file-mtime. When I run rsync to download it, it stores the file with the S3 timestamp and I'm curious if there is a way to have it stored with the file-mtime timestamp?

Run the command 'rclone version' and share the full output of the command.

rclone v1.71.0

  • os/version: Microsoft Windows 11 Enterprise 23H2 23H2 (64 bit)
  • os/kernel: 10.0.22631.5768 (x86_64)
  • os/type: windows
  • os/arch: amd64
  • go/version: go1.25.0
  • go/linking: static
  • go/tags: cmount

Which cloud storage system are you using? (eg Google Drive)

AWS S3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync S3remote:s3/path/folder \\server\folder --multi-thread-streams 16 --checksum --combined log.txt --progress

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[S3remote]
type = s3
provider = AWS
access_key_id = XXX
secret_access_key = XXX
region = eu-west-1
acl = private
storage_class = STANDARD
bucket_acl = private

A log from the command that you were trying to run with the -vv flag

Paste  log here
2

You can transform metadata using the metadata mapper

I think if you transform the file-mtime metadata into the mtime metadata then rclone should pick it up. You'll need to write a simple script to do this - there are examples in the docs.

The SMB backend does not support matadata in general but I think the mtime medatada will be used to set the modification time on files. I didn't check that though, so I'd check that first with –metadata-set mtime=xxx first

3

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.