--track-renames doesn't seem to work with Wasabi S3?

Help and Support
1

What is the problem you are having with rclone?

I've set up an rclone sync between gdrive and Wasabi S3 on an f1.micro instance. I need to rename the files at some point (doesn't matter when, either in transit or after sync), but once I do and run sync again, it deletes the renamed file and re-syncs with the previous name. I need the name to stay the same.

I can't change the filenames at the source since it's a shared drive that I don't own. I need to change the filenames as I'm planning to use this S3 bucket to serve content for a website, and there are # characters in the filenames which doesn't play nice with S3.

What is your rclone version (output from rclone version)

rclone v1.52.1
- os/arch: linux/amd64
- go version: go1.14.4

Which OS you are using and how many bits (eg Windows 7, 64 bit)

PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian

Which cloud storage system are you using? (eg Google Drive)

Google Drive, Wasabi

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync --drive-shared-with-me gdrive:"folder"/"sub-folder" wasabi:bucket/"folder" --track-renames

The rclone config contents with secrets removed.

[wasabi]
type = s3
provider = Wasabi
access_key_id = xxx
secret_access_key = xxx
endpoint = s3.wasabisys.com
location_constraint = us-east-1
acl = private
[gdrive]
type = drive
client_id = xxx
client_secret = xxx
scope = drive
token = {"access_token":"xxx","token_type":"Bearer","refresh_token":"xxx"}
root_folder_id = xxx

A log from the command with the -vv flag

If required please let me know.
2

hello and welcome to the forum,
i use wasabi and i do not have a problem with filenames having #
https://rclone.org/s3/#restricted-filename-characters

2020/06/23 20:22:34 DEBUG : rclone: Version "v1.52.0" starting with parameters ["rclone.exe" "copy" ".\hash" "bar:hashhashhash" "-vv"]
2020/06/23 20:22:34 DEBUG : S3 bucket hashhashhash: Waiting for transfers to finish
2020/06/23 20:22:35 DEBUG : #######.#####: MD5 = 59fb38dbd2e7c8461be5422a4e12cacb OK
2020/06/23 20:22:35 INFO  : #######.#####: Copied (new)
2020/06/23 20:22:35 INFO  : 
Transferred:            1 / 1, 100%

rclone.exe ls bar:hashhashhash 
        5 #######.#####
1 Like
3

Hey thanks!

That's really weird, maybe it's because I'm on the Wasabi trial (testing out to see if it'll be any good for this purpose), but whenever there's a hash mark in the name of a file, e.g. Filename #test, it gives me this error when I try to view the file by its URL:

<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Key>Filename </Key>
<RequestId>2EEWEFWEFC2909E5292DFC</RequestId>
<HostId>kQdsfasdf6csOY8BAy7QkCmLi0mpk4jmPbhyEv9MaZNNjGDvH+Szl7s2xy5+iMD9aBf/b4k8K/gfOY+yse+</HostId>
</Error>

The filename is always cut off starting from where the hashmark is :confused:

4

does the filename get cut off from
where the hashmark is
or
where the space character is

add -vv to the comamnd for debug output.
you need to run the command with debug info and post both the command and output.

add --dry-run to test without copying.

1 Like
5

From where the hashmark is, i.e. if the filename is Filename #test, the error message in browser will show <key>Filename </key> with the space before the hash. Verbose output:

rclone sync --drive-shared-with-me gdrive:"folder"/"subfolder" wasabi:bucketname/"folder + name" --track-renames -vv
2020/06/23 23:57:33 DEBUG : rclone: Version "v1.52.1" starting with parameters ["rclone" "sync" "--drive-shared-with-me" "
gdrive:folder/subfolder" "wasabi:bucketname/folder + name" "--
track-renames" "-vv"]
2020/06/23 23:57:33 DEBUG : Using config file from "/home/jonathan_alumbaugh/.config/rclone/rclone.conf"
2020/06/23 23:57:36 NOTICE: Filename #311.2 - other stuff: Duplicate object found in source - ignoring
2020/06/23 23:57:36 NOTICE: Filename #327.1 - other stuff: Duplicate object found in source - ignoring
2020/06/23 23:57:36 INFO  : S3 bucket bucketname path Filename folder: Making map for --track-renames
2020/06/23 23:57:36 INFO  : S3 bucket bucketname path Filename folder: Finished making map for --track-renames
2020/06/23 23:57:36 DEBUG : S3 bucket bucketname path Filename folder: Waiting for checks to finish
2020/06/23 23:57:36 DEBUG : Filename #0.40 - other stuff.mp4: Size and modification time the same (differ by 0
s, within tolerance 1ms)
2020/06/23 23:57:36 DEBUG : Filename #0.40 - other stuff.mp4: Unchanged skipping
2020/06/23 23:57:36 DEBUG : Filename #0.41 - other stuff.mp4: Size and modification time the same (differ by 0s, wi
thin tolerance 1ms)
2020/06/23 23:57:36 DEBUG : Filename #0.41 - other stuff.mp4: Unchanged skipping
2020/06/23 23:57:36 DEBUG : Filename #1 - other stuff.mp4: Size and modification time the same (differ by 0s, within t
olerance 1ms)
2020/06/23 23:57:36 DEBUG : Filename #1 - other stuff.mp4: Unchanged skipping
2020/06/23 23:57:36 DEBUG : Filename #10 - other stuff.mp4: Size and modification time the same (differ by 0s, wit
hin tolerance 1ms)
2020/06/23 23:57:36 DEBUG : Filename #10 - other stuff.mp4: Unchanged skipping
2020/06/23 23:57:36 DEBUG : Filename #100 - other stuff.mp4: Size and modification time the same (differ by 0s, wit
hin tolerance 1ms)
2020/06/23 23:57:37 DEBUG : Filename #NA3 - other stuff.mp4: Size and modification time the same (differ by 0s, within tolerance 1ms)
2020/06/23 23:57:37 DEBUG : Filename #NA3 - other stuff.mp4: Unchanged skipping
2020/06/23 23:57:37 DEBUG : S3 bucket bucketname path Filename folder: Waiting for renames to finish
2020/06/23 23:57:37 DEBUG : S3 bucket bucketname path Filename folder: Waiting for transfers to finish
2020/06/23 23:57:37 DEBUG : Waiting for deletions to finish
2020/06/23 23:57:37 INFO  : There was nothing to transfer
2020/06/23 23:57:37 INFO  : 
Transferred:             0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:               676 / 676, 100%
Elapsed time:         0.5s

2020/06/23 23:57:37 DEBUG : 35 go routines active
6

i could be wrong but that is weird looking usage of quotes
https://rclone.org/docs/#linux-osx

try quoting the entire remote name
"gdrive:folder/subfolder"
and
"wasabi:bucketname/folder + name"

1 Like
7

Thanks, using this doesn't seem to make a difference, like if I sync, rename a file, and sync again, it'll undo the change:

rclone sync --drive-shared-with-me "gdrive:folder/subfolder" "wasabi:bucket/folder" --track-renames
8

Did some more reading, found a thread which clears up my misunderstanding of --track-renames. It doesn't appear that --track-renames is meant to function how I'd previously understood, so I wonder what my options are to preserve changed names between syncs. Should I open a new issue and delete this one, since the title isn't correct?

9

A new post would probably be best. Try to describe what you would like to do and what you think --track-renames should do in it.

10

Will do, thank you @ncw and @asdffdsa! My apologies for my confusion.

1 Like
11

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.