Rclone stops(instead of moving on to other files) after a series of PathIsTooLong error messages on Termux

What is the problem you are having with rclone?

I tried running rclone in Termux, both on my Android phone and Tab. I tried to copy the entire contents of their storages (/sdcard and /storage/0114-6400 of phone and /sdcard of the tab) in to respective three separate encrypted remotes on OneDrive.

  1. Both devices fail to copy some files because of file paths that are too long, which is expected. However, instead of skipping those files and moving on to other files, copying stops after some of those PathIsTooLong error messages. Note that all the error messages are PathIsTooLong error messages (except for the last two of course). This issue happens every time I try. (The directory is uploaded to some extent. So the device immediately starts listing the files with long paths and then, the copying comes to a halt(i.e., the $ sign of the terminal is displayed). The directory is NOT completely copied yet. The source and destination folder sizes are totally different on cloud and on local storage.The same thing happens with the the internal storage on phone(/sdcard), only with a different number of error messages. Those too are all PathIsTooLong messages.

  2. Not sure if this is an error with rclone or not, but when I tried copying the external storage to OneDrive, the copying works fine, except that I found that about 5GB of storage were reduced according to the file manager after the command to copy was executed. I terminated Termux before the copying was complete. Could these possibly be junk/temporary files that were deleted? I can't figure out what is missing(if anything is missing, that is).

  3. Again, not sure if this an issue with rclone, but I found some files that were supposed to be copied were somehow in data/data/com.termux/files/home directory when I ran an 'ls' command. Some files from my source(source for copying) and destination(destination to be copied to) were listed. This is what the 'find' command produces.

~ $ find
.
./.termux
./.termux/termux.properties
./.termux/termux.properties.save
./.bash_history
./nano.20796.save
./testii.py
./.viminfo
./.config
./.config/rclone
./.config/rclone/rclone.conf
./storage
./storage/shared
./storage/downloads
./storage/dcim
./storage/pictures
./storage/music
./storage/movies
./storage/external-1
./nano.31176.save
./UniOneDriveEncrypted
./UniOneDriveEncrypted/temp1.xlsx

'Storage' folder has much more depth in terms of subfolders. UniOneDriveEncrypted is the remote name. 'temp1.xlsx' was actualy initially uploaded for testing purposes, and since was deleted. It was encrypted so that wasn't the name that appeared when it was uploaded.

Is this to be expected? Should I care or not? Is there a way to clear the data? Can I just delete Data in App info? Any advice is appreciated.

Run the command 'rclone version' and share the full output of the command.

  1. For the tab:
    ~ $ rclone version
    rclone 1.64.2-termux
  • os/version: unknown
  • os/kernel: 4.4.111-17423164 (armv8l)
  • os/type: android
  • os/arch: arm (ARMv7 compatible)
  • go/version: go1.21.3
  • go/linking: dynamic
  • go/tags: noselfupdate
  1. For the phone:
    ~ $ rclone version
    rclone 1.64.2-termux
  • os/version: unknown
  • os/kernel: 4.4.177-22392479 (aarch64)
  • os/type: android
  • os/arch: arm64 (ARMv8 compatible)
  • go/version: go1.21.3
  • go/linking: dynamic
  • go/tags: noselfupdate

Which cloud storage system are you using? (eg Google Drive)

OneDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

  1. In phone
rclone copy /sdcard PhoneToUniOneDriveEncrypted
rclone copy /storage/0114-6400 PhoneSDToUniOneDriveEncrypted
  1. In tab
rclone copy /sdcard UniOneDriveEncrypted

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

  1. For the tab:
~ $ rclone config redacted
[UniOneDrive]
type = onedrive
token = XXX
drive_id = XXX
drive_type = business

[UniOneDriveEncrypted]
type = crypt
remote = UniOneDrive:/tabbackup
password = XXX
### Double check the config for sensitive info before posting publicly
  1. For the phone:
~ $ rclone config redacted
[PhoneSDToUniOneDriveEncrypted]
type = crypt
remote = UniOneDrive:phonesdbackup
password = XXX

[PhoneToUniOneDriveEncrypted]
type = crypt
remote = UniOneDrive:phonebackup
password = XXX

[UniOneDrive]
type = onedrive
token = XXX
drive_id = XXX
drive_type = business
### Double check the config for sensitive info before posting publicly

A log from the command that you were trying to run with the -vv flag

  1. The last part of the error message is here.
2023/11/05 16:10:00 ERROR : WhatsApp/Media/WhatsApp Documents/******.pdf: Failed to copy: invalidRequest: pathIsTooLong: Path is too long

2023/11/05 16:10:02 ERROR : WhatsApp/Media/WhatsApp Documents/******.pdf: Failed to copy: invalidRequest: pathIsTooLong: Path is too long

2023/11/05 16:10:03 ERROR : WhatsApp/Media/WhatsApp Documents/******.pdf: Failed to copy: invalidRequest: pathIsTooLong: Path is too long

2023/11/05 16:13:45 ERROR : Android/data/com.sec.android.app.myfiles/files/01EAUZ2KQN4RD2LUEG2BELRKGZXF22ARPM/1654771842/******.pdf: Failed to copy: invalidRequest: pathIsTooLong: Path is too long

2023/11/05 16:13:46 ERROR : Android/data/com.sec.android.app.myfiles/files/01EAUZ2KT2J443OWZFHFA2SSNI4SX5WPZD/1643292510/******.pdf: Failed to copy: invalidRequest: pathIsTooLong: Path is too long

2023/11/05 16:15:48 ERROR : Can't retry any of the errors - not attempting retries

2023/11/05 16:15:48 Failed to copy with 103 errors: last error was: invalidRequest: pathIsTooLong: Path is too long

Any help is greatly appreciated.

Onedrive allows only max 400 characters per path - when you add names' encryption and use base32 then it becomes something like 250 or less. It is not fix number as e.g. very short names have to be encrypted using minimum encrypted length - read rclone crypt doc if you are interested about details.

Practical solution for Onedrive is not to use default base32 names crypt encoding but switch to base32768 - it will give you much better max path length.

But even then max path length will be shorter than allowed on Android - you have to make sure you do not use too long names and to deep directories' structures.

Hi. Thank you for your reply. Yes that is the case but isn't rclone supposed to move on to other files that do not have a long path instead of quitting? Does changing the file names fix the issue altogether?

Not sure TBH but agree that it would be more logical.

Try with --ignore-errors flag.

Other rather clever approach is here:

Thanks. :smile: I will give these a try

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.