What is the problem you are having with rclone?
No problem. A question as preventive maintenance.
We are beginning upload of several million files to a Dropbox remote wrapped in a crypt remote in rclone. We are using filename_encoding and have a number of files with very long filenames.
A few hundred of the longest named files generate filenames which are too long for Dropbox with either base32 or base64 encoding (the Failed to copy: file name too long
error).
An upload test of the same files using base32768 is successful. Excellent so far.
Digging through rclone and other forums there is some discussion of Dropbox character limitations for Unicode. Indeed running rclone test info
shows 4-byte character incompatibility.
Yet I don't see reference to specific/repeatable problems with base32768 encoding of filenames and Dropbox.
maxFileLength = 255 // for 1 byte unicode characters
maxFileLength = 255 // for 2 byte unicode characters
maxFileLength = 255 // for 3 byte unicode characters
maxFileLength = -1 // for 4 byte unicode characters
canWriteUnnormalized = true
canReadUnnormalized = true
canReadRenormalized = true
canStream = true
Questions:
Are there known incompatibilities with a crypt remote using filename_encoding = base32768
and an underlying Dropbox remote?
If the source filenames contain no emoji or 4-byte Unicode characters, might base32768 nonetheless create a 4-byte character when encoding?
On the rclone.org/overview
page you do mention Some cloud storage systems might have restrictions on the characters that are usable in file or directory names. When rclone detects such a name during a file upload, it will transparently replace the restricted characters with similar looking Unicode characters.
. But I'm not sure if that applies also to a remote wrapped in a crypt remote.
We've run a test uploading 10,000 or so files with base32768 encoding and so far not run into any issues. But there are in total several million files, thus would prefer to not hit it at the 3,888,888th file.
If you have no report of specific incompatibilities of dropbox + crypt w filename_encoding = base32768 then brilliant. We will forge ahead.
As always, thank you for the amazing tool and the excellent support forum.
Run the command 'rclone version' and share the full output of the command.
rclone v1.63.0-beta.6938.243bcc9d0
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-67-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.2
- go/linking: static
- go/tags: none
Which cloud storage system are you using? (eg Google Drive)
Dropbox Business , primarily team folders dropbox_remote:/
The command you were trying to run (eg rclone copy /tmp remote:tmp
)
rclone copy local:/path/to/folder dropbox_crypt:/path/to/folder -vvP
The rclone config contents with secrets removed.
[dropbox_crypt]
type = crypt
password = blah
remote = dropbox:
directory_name_encryption = false
filename_encoding = base32768```
#### A log from the command with the `-vv` flag
No errors with base32768. With base32 or base64 the error for excessively long files is
2023/04/05 09:50:05 Failed to copy with 10 errors: last error was: file name too long