Convert Filename Encoding

Help and Support
1

What is the problem you are having with rclone?

I made a mistake and set up a Onedrive crypt without filename encoding set, so I am encountering "pathIsTooLong" errors. If I simply add filename_encoding = base32768, of course my data doesn't show up, since it wasn't created with that encoding.

Is there a way for me to change the filename encoding without re-uploading all my data? It's about 21,000 objects, 310 GB.

Run the command 'rclone version' and share the full output of the command.

rclone v1.58.0
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-1017-raspi (aarch64)
- os/type: linux
- os/arch: arm64
- go/version: go1.17.8
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Onedrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync -P --size-only --exclude "zz Old Files/**" --exclude "._*" $ROOT_FOLDER/$SUB_FOLDER/ $RCLONE_REMOTE:/$SUB_FOLDER/

The rclone config contents with secrets removed.

[onedrive-deu]
type = onedrive
token = zzz
drive_id = zzz
drive_type = personal

[onecrypt]
type = crypt
remote = onedrive-deu:linux
password = zzz
password2 = zzz

A log from the command with the -vv flag

2022-11-09 19:17:25 ERROR : [really long path name]: Failed to copy: invalidRequest: pathIsTooLong: Path exceeds maximum length
2

hello and welcome to the forum,

https://rclone.org/crypt/#crypt-server-side-across-configs
"to change file name encryption type without re-uploading all the data. Just make two crypt backends pointing to two different directories with the single changed parameter and use rclone move to move the files between the crypt remotes"

so somethihg like

[onedrive-deu]
type = crypt
remote = onedrive-deu:linux
password = zzz
password2 = zzz

[onecrypt_dest]
type = crypt
remote = onedrive-deu:dest
password = zzz
password2 = zzz
filename_encoding = base32768

and as a test, on a small subdir, tho need to remove --dry-run to actually copy files.

rclone copy --dry-run --crypt-server-side-across-configs onedrive-deu:subdir onecrypt_dest:subdir
rclone ls onecrypt_dest:subdir
rclone cryptcheck onedrive-deu:subdir onecrypt_dest:subdir
3

That is a really cool feature, seems to be exactly what I needed, thanks for pointing it out! Running it now.

4

For some reason this is taking about as long as the initial upload. Seems like it's not doing it server side, for some reason. For reference:

rclone move --progress --crypt-server-side-across-configs onecrypt-old: onecrypt:

At one point I got this error, but it seemed to be having issues before that:

2022-11-12 05:29:13 ERROR : UnknownError: : upload chunks may be taking too long - try reducing --onedrive-chunk-size or decreasing --transfers
5
  1. add -vv for debug output
  2. run the command on a single file and post the entire debug output.
6 
rclone copy -vv --progress --crypt-server-side-across-configs onecrypt-old:test.txt onecrypt:test.txt

2022/11/12 11:24:17 DEBUG : rclone: Version "v1.58.0" starting with parameters ["rclone" "copy" "-vv" "--progress" "--crypt-server-side-across-configs" "onecrypt-old:test.txt" "onecrypt:test.txt"]
2022/11/12 11:24:17 DEBUG : Creating backend with remote "onecrypt-old:test.txt"
2022/11/12 11:24:17 DEBUG : Using config file from "/home/lightsong/.config/rclone/rclone.conf"
2022/11/12 11:24:17 DEBUG : onecrypt-old: detected overridden config - adding "{Db_Y9}" suffix to name
2022/11/12 11:24:17 DEBUG : Creating backend with remote "onedrive-deu:linux/u6ii3cajgci4k7ica51fol0r4c"
2022/11/12 11:24:19 DEBUG : fs cache: adding new entry for parent of "onedrive-deu:linux/u6ii3cajgci4k7ica51fol0r4c", "onedrive-deu:linux"
2022/11/12 11:24:19 DEBUG : fs cache: adding new entry for parent of "onecrypt-old:test.txt", "onecrypt-old{Db_Y9}:test.txt"
2022/11/12 11:24:19 DEBUG : Creating backend with remote "onecrypt:test.txt"
2022/11/12 11:24:19 DEBUG : onecrypt: detected overridden config - adding "{Db_Y9}" suffix to name
2022/11/12 11:24:19 DEBUG : Creating backend with remote "onedrive-deu:linux/鼲滌僐墤睒垥穻㠿"
2022/11/12 11:24:20 DEBUG : fs cache: renaming cache item "onecrypt:test.txt" to be canonical "onecrypt{Db_Y9}:test.txt"
2022-11-12 11:24:22 DEBUG : test.txt: Need to transfer - File not found at Destination
2022-11-12 11:24:23 DEBUG : 鼲滌僐墤睒垥穻㠿: Starting multipart upload
2022-11-12 11:24:24 DEBUG : 鼲滌僐墤睒垥穻㠿: Uploading segmen t 0/74 size 74
2022-11-12 11:24:24 DEBUG : test.txt: sha1 = cf23c0376cc961235a0ed570e9c05d08f36fd8f8 OK
2022-11-12 11:24:24 INFO  : test.txt: Copied (new)
Transferred:             74 B / 74 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         7.1s
2022/11/12 11:24:24 INFO  :
Transferred:             74 B / 74 B, 100%, 0 B/s, ETA -
Transferred:            1 / 1, 100%
Elapsed time:         7.1s

2022/11/12 11:24:24 DEBUG : 9 go routines active
7

1 update rclone - rclone selfupdate
2. remove --crypt-server-side-across-configs
3. add --server-side-across-configs

rclone move -vv onecrypt_encoding_source: onecrypt_encoding_dest: --server-side-across-configs 
DEBUG : rclone: Version "v1.60.0" starting with parameters ["C:\\data\\rclone\\rclone" "move" "-vv" "onecrypt_encoding_source:" "onecrypt_encoding_dest:" "--server-side-across-configs"]
INFO  : file.ext: Moved (server-side)
INFO  : There was nothing to transfer
INFO  : 
Transferred:   	    1.382 MiB / 1.382 MiB, 100%, 0 B/s, ETA -
Checks:                 1 / 1, 100%
Renamed:                1

[onecrypt_encoding_source]
type = crypt
remote = onedrive:cryptsource
password = redacted
password2 = redacted

[onecrypt_encoding_dest]
type = crypt
remote = onedrive:cryptdest
password = redacted
password2 = redacted
filename_encoding = base32768
8

Updating rclone and switching to --server-side-across-configs worked great, thanks!

9

sorry for the confusion, as i have never tried to change filename encoding.

the rclone docs are very clear, use --crypt-server-side-across-configs
but once you posted the debug output, i did some quick testing for you

10

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.