Problem with files-from option

What is the problem you are having with rclone?

When I try to use the command, the files listed in rclone_storage_sync.txt are not transferred, and the 'events' directory is excluded (the files do not exist remotely).

This is an example of file /tmp/rclone_storage_sync.txt:

/var/www/example.it/production/htdocs/public/storage/events/b5c4e8/uploads/66f4f673b5eab_thumb.jpeg
/var/www/example.it/production/htdocs/public/storage/events/b5c4e8/uploads/66f4f6663ac97_thumb.jpg
/var/www/example.it/production/htdocs/public/storage/events/b5c4e8/uploads/66f4f668e64a9.jpeg
/var/www/example.it/production/htdocs/public/storage/events/b5c4e8/uploads/66f4f66a5217a_thumb.jpg
/var/www/example.it/production/htdocs/public/storage/events/b5c4e8/uploads/66f4f67384e5c.jpeg
/var/www/example.it/production/htdocs/public/storage/events/b5c4e8/uploads/66f4f6726cb24.mp4
/var/www/example.it/production/htdocs/public/storage/events/b5c4e8/uploads/66f4f665f044f.jpeg

Run the command 'rclone version' and share the full output of the command.

rclone v1.68.1

  • os/version: debian 12.5 (64 bit)
  • os/kernel: 6.1.0-10-amd64 (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.23.1
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Contabo S3 object storage

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copy --files-from /tmp/rclone_storage_sync.txt /var/www/example.it/production/htdocs/public/storage PB_CONTABO_STORAGE:bucket-it/ -vv

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[PB_CONTABO_STORAGE]
type = s3
provider = Other
access_key_id = XXX
secret_access_key = XXX
endpoint = eu2.contabostorage.com
acl = private

[PB_DRIVE]
type = drive
client_id = XXX
client_secret = XXX
scope = drive
token = XXX
team_drive = 

A log from the command that you were trying to run with the -vv flag

2024/09/26 16:57:28 DEBUG : rclone: Version "v1.68.1" starting with parameters ["rclone" "copy" "--files-from" "/tmp/rclone_storage_sync.txt" "/var/www/example.it/production/htdocs/public/storage" "PB_CONTABO_STORAGE:bucket-it/" "-vv"]
2024/09/26 16:57:28 DEBUG : Creating backend with remote "/var/www/example.it/production/htdocs/public/storage"
2024/09/26 16:57:28 DEBUG : Using config file from "/var/www/.config/rclone/rclone.conf"
2024/09/26 16:57:28 DEBUG : Creating backend with remote "PB_CONTABO_STORAGE:bucket-it/"
2024/09/26 16:57:28 DEBUG : fs cache: renaming cache item "PB_CONTABO_STORAGE:bucket-it/" to be canonical "PB_CONTABO_STORAGE:bucket-it"
2024/09/26 16:57:28 DEBUG : events: Excluded
2024/09/26 16:57:29 DEBUG : events: Excluded
2024/09/26 16:57:29 DEBUG : .gitignore: Excluded (FilesFrom Filter)
2024/09/26 16:57:29 DEBUG : .gitignore: Excluded
2024/09/26 16:57:29 DEBUG : S3 bucket bucket-it: Waiting for checks to finish
2024/09/26 16:57:29 DEBUG : S3 bucket bucket-it: Waiting for transfers to finish
2024/09/26 16:57:29 INFO  : There was nothing to transfer
2024/09/26 16:57:29 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Elapsed time:         0.1s

welcome to the forum,

when using filters, the paths are always relative to the root of the source.

in your case, the source is /var/www/example.it/production/htdocs/public/storage, so i think

/var/www/example.it/production/htdocs/public/storage/events/b5c4e8/uploads/66f4f673b5eab_thumb.jpeg
should be
/events/b5c4e8/uploads/66f4f673b5eab_thumb.jpeg

does not matter, as filters work on the source only,
so the safest way to test is rclone ls else rclone copy --dry-run

It worked, I hadn't thought about the path issue. Thank you so much for your help! :grinning:

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.