How do I cut filenames on mounts where the filename is too long

Help and Support
1

What is the problem you are having with rclone?

Is it somehow possible to cut filenames that are too long in a mount? Currently, files which are too long end up in a log message and don’t work.

I started messing around with a small script that truncates files correctly in vfs and vfsMetadata directory, however I am not sure whether they are uploaded or how to trigger that using the following script:

#!/bin/bash

# Set the maximum filename length in bytes
MAX_LENGTH=143

# Directory to traverse
DIRECTORY="/mnt/ssd/rclone/drive/"  # Change this to the path of your directory

# Function to get the byte size of a character
get_byte_size() {
    local char="$1"
    local byte_size
    byte_size=$(echo -n "$char" | wc -c)
    echo "$byte_size"
}

# Find all files in the directory and process each file
find "$DIRECTORY" -type f | while read -r FILE; do
    FILENAME=$(basename "$FILE")
    DIRNAME=$(dirname "$FILE")

    # Initialize variables
    total_byte_size=0
    truncate_index=${#FILENAME}

    # Loop through each character from the end and accumulate byte size
    for (( i=${#FILENAME}-1; i>=0; i-- )); do
        char="${FILENAME:i:1}"
        byte_size=$(get_byte_size "$char")
        total_byte_size=$((total_byte_size + byte_size))

        # If the total byte size exceeds the max length, break the loop
        if [ "$total_byte_size" -gt "$MAX_LENGTH" ]; then
            truncate_index=$((i + 1))
            break
        fi
    done

    # If the total byte size exceeds the max length, truncate the front
    if [ "$total_byte_size" -gt "$MAX_LENGTH" ]; then
        NEW_FILENAME="${FILENAME:truncate_index}"
        mv "$FILE" "$DIRNAME/$NEW_FILENAME"
        echo "Renamed: $FILENAME -> $NEW_FILENAME"
    fi
done

These are my limits:

maxFileLength = 143 // for 1 byte unicode characters
maxFileLength = 71 // for 2 byte unicode characters
maxFileLength = 47 // for 3 byte unicode characters
maxFileLength = 35 // for 4 byte unicode characters

So ideally, I would trim name to 143 bytes of characters. Is there a possibility to run this script on any file? Or does rclone have anything implemented?

Run the command 'rclone version' and share the full output of the command.

rclone v1.71.0
- os/version: fedora 42 (64 bit)
- os/kernel: 6.15.10-200.fc42.x86_64 (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.24.6
- go/linking: dynamic
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

SFTP

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount \
    --rc \
    --config=/home/test/.config/rclone/rclone.conf \
    --allow-other \
    --cache-dir /mnt/ssd/rclone/drive \
    --vfs-cache-mode full \
    --vfs-cache-max-size 500G \
    drive: /home/test/rclone/drive

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

[drive]
type = crypt
remote = myRemote:drive
password = XXX
[myRemote]
type = sftp
host = XXX
user = XXX
port = 53211
key_file = ~/.ssh/myRemote
shell_type = unix
md5sum_command = md5sum
sha1sum_command = sha1sum
2

welcome to the forum,

if the backend is gdrive, then
"For cloud storage systems with case sensitive file names (e.g. Google Drive), base64 can be used to reduce file name length"

but that would not always work for other backends

3

No the backend is not gdrive, maybe I didn’t quite get what a backend exactly is, excuse me. I am using SFTP to connect to a StorageBox from hostbrr.

4

yeah, i got that wrong.

yeah, i love my hetzner storage box.

just curios, you remote is using port = 53211 ?

Hetzner Storage Boxes are supported through the SFTP backend on port 23

my remote looks like

[sbox01_sftp]
type = sftp
host = redacted.your-storagebox.de
port = 23
key_file = d:\data\c\combined\hetzner\keys\id_ed25519
user = redacted
shell_type = unix
md5sum_command = md5 -r
sha1sum_command = sha1 -r
5

No I am not using hetzner, I am using a Storage Box offer from HostBrr, I think the server is hosted at hetzner but other than that they are unrelated. I got quite a generous offer so I don’t plan on switching, I rather have my filenames shortened :smiley:

6

rclone convmv with --name-transform truncate_bytes_keep_extension=143

7

As Sonarr and Radarr by default put metadata in the back of the filename, I’d rather have rclone trim from back instead of from the front. Could I reuse my script?

What is given and what should the script yield when it should run in a mount?

Is it as simple as:

INPUT(as in argument): filename

OUTPUT(as in return): new-filename?

8

As far as I understood rclone convmv does not work with a mount?

Alternatively, is my current approach feasible?

A script renames files inside the vfs and vfsMetadata directories and shorten their filenames.

I think this is something that should run regularly before caches expire, although I am not very sure about the upload process.

9

In my opinion, you are asking for trouble by trying to manually rename files in your underlying vfs and vfsMetadata directories directly.

They are different things. The workflow I think makes the most sense is to first rclone convmv to rename the files on the remote. Then, mount your remote which already has the correct names.

You could, in theory, mount and then spin up a second instance of rclone to convmv the mounted path. But that won't help you if the problem is that the filenames are precluding them from even appearing in the mount.

You could try playing around with the --name-transform trimprefix=XXXX option in convmv. But it may require some scripting if the prefix is not consistent.

10

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.