I am attempting to copy a section of a file tree from a computer into an S3 bucket. I have copying to the bucket fine but using --include to copy just part of the tree is confusing me.
I have a tree:
/volume1/.srv/unifi-protect/video/2024/11/01/
/volume1/.srv/unifi-protect/video/2024/11/02/
/volume1/.srv/unifi-protect/video/2024/11/03/
and so on.
Here is the command I'm using
rclone --config ${PWD}/rclone.conf --dry-run \
--verbose copy \
/volume1/.srv/unifi-protect/video/ \
--include "*/2024/11/0*" \
nvr-backup:/nvr-backup-340752817366/
I get INFO: There was nothing to transfer every time
I have tried it with a slash at the end of the include string. I also tried it with the entire path at the left of the include string. Also a bunch of similar changes.
Note that:
rclone --config ${PWD}/rclone.conf --dry-run \
--verbose copy \
/volume1/.srv/unifi-protect/video/2024/11/01 \
nvr-backup:/nvr-backup-340752817366/
correctly shows all the files in the directory would be copied
Run the command 'rclone version' and share the full output of the command.
rclone version
rclone v1.69.0
- os/version: debian 11.8 (64 bit)
- os/kernel: 4.19.152-alpine-unvr (aarch64)
- os/type: linux
- os/arch: arm64 (ARMv8 compatible)
- go/version: go1.23.4
- go/linking: static
- go/tags: none
Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
-→
Yes, on the latest
Which cloud storage system are you using? (eg Google Drive)
S3
The command you were trying to run (eg rclone copy /tmp remote:tmp)
rclone --config ${PWD}/rclone.conf --dry-run \
--verbose copy \
/volume1/.srv/unifi-protect/video/ \
--include "*/2024/11/0*" \
nvr-backup:/nvr-backup-340752817366/
Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.
[nvr-backup]
type = s3
provider = AWS
env_auth = false
region = ap-southeast-2
location_constraint = ap-southeast-2
acl = private
access_key_id = XXX
A log from the command that you were trying to run with the -vv flag
rclone --config ${PWD}/rclone.conf --dry-run --verbose copy /volume1/.srv/unifi-protect/video/2024/11/01 --include "*2024/11/0*/" nvr-backup:/nvr-backup-340752817366/2024/10 -vv
2025/02/06 15:15:32 DEBUG : rclone: Version "v1.69.0" starting with parameters ["rclone" "--config" "/home/nvr-backup/rclone.conf" "--dry-run" "--verbose" "copy" "/volume1/.srv/unifi-protect/video/2024/11/01" "--include" "*2024/11/0*/" "nvr-backup:/nvr-backup-340752817366/2024/10" "-vv"]
2025/02/06 15:15:32 DEBUG : Creating backend with remote "/volume1/.srv/unifi-protect/video/2024/11/01"
2025/02/06 15:15:32 DEBUG : Using config file from "/home/nvr-backup/rclone.conf"
2025/02/06 15:15:32 DEBUG : Creating backend with remote "nvr-backup:/nvr-backup-340752817366/2024/10"
2025/02/06 15:15:32 DEBUG : fs cache: renaming cache item "nvr-backup:/nvr-backup-340752817366/2024/10" to be canonical "nvr-backup:nvr-backup-340752817366/2024/10"
2025/02/06 15:15:32 DEBUG : S3 bucket nvr-backup-340752817366 path 2024/10: Waiting for checks to finish
2025/02/06 15:15:32 DEBUG : S3 bucket nvr-backup-340752817366 path 2024/10: Waiting for transfers to finish
2025/02/06 15:15:32 INFO : There was nothing to transfer
2025/02/06 15:15:32 NOTICE:
Transferred: 0 B / 0 B, -, 0 B/s, ETA -
Elapsed time: 0.0s
2025/02/06 15:15:32 DEBUG : 5 go routines active