Onedrive directories and special characters

[DataMovr]

What is the problem you are having with rclone?

I can't get rclone to perform file/directory operations for directories that begin with the ~ character. I'm ultimately just trying to remove these directories that start with ~ and using purge, delete, or rmdir commands all provide a directory not found error.

Run the command 'rclone version' and share the full output of the command.

C:\ProgramData\chocolatey\bin>rclone version
rclone v1.60.1

• os/version: Microsoft Windows 11 Pro 21H2 (64 bit)
• os/kernel: 10.0.22000.1219 (x86_64)
• os/type: windows
• os/arch: amd64
• go/version: go1.19.3
• go/linking: static
• go/tags: cmount

YES

Which cloud storage system are you using? (eg Google Drive)

Microsoft OneDrive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone purge LLOD:~dir2 -vv

The rclone config contents with secrets removed.

[LLOD]
type = onedrive
token = {"access_token":"sECreTtokenHERe","expiry":"2022-12-13T22:24:52.7932792-06:00"}
drive_id = b!thisisnottherealdriveidlalalalala
drive_type = business

A log from the command with the -vv flag

...

C:\ProgramData\chocolatey\bin>rclone purge LLOD:~dir2 -vv
2022/12/13 21:32:12 DEBUG : rclone: Version "v1.60.1" starting with parameters ["C:\ProgramData\chocolatey\lib\rclone.portable\tools\rclone-v1.60.1-windows-amd64\rclone.exe" "purge" "LLOD:~dir2" "-vv"]
2022/12/13 21:32:12 DEBUG : Creating backend with remote "LLOD:~dir2"
2022/12/13 21:32:12 DEBUG : Using config file from "C:\Users\RM\AppData\Roaming\rclone\rclone.conf"
2022/12/13 21:32:13 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
2022/12/13 21:32:13 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
2022/12/13 21:32:14 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
2022/12/13 21:32:14 DEBUG : 3 go routines active
2022/12/13 21:32:14 Failed to purge: directory not found

...

C:\ProgramData\chocolatey\bin>rclone lsd LLOD: -vv
2022/12/13 21:17:05 DEBUG : rclone: Version "v1.60.1" starting with parameters ["C:\ProgramData\chocolatey\lib\rclone.portable\tools\rclone-v1.60.1-windows-amd64\rclone.exe" "lsd" "LLOD:" "-vv"]
2022/12/13 21:17:05 DEBUG : Creating backend with remote "LLOD:"
2022/12/13 21:17:05 DEBUG : Using config file from "C:\Users\RM\AppData\Roaming\rclone\rclone.conf"
-1 2022-12-12 22:12:33 17 20180409
-1 2022-12-12 22:12:34 0 20180411
-1 2020-07-15 22:35:14 257 ~dir1
-1 2020-07-15 22:32:29 8 ~dir2
-1 2020-07-15 22:29:58 149 ~dir3
-1 2020-07-28 02:41:51 717 ~dir4
2022/12/13 21:11:20 DEBUG : 3 go routines active

...

C:\ProgramData\chocolatey\bin>rclone ls LLOD:~dir2 -vv
2022/12/13 21:22:24 DEBUG : rclone: Version "v1.60.1" starting with parameters ["C:\ProgramData\chocolatey\lib\rclone.portable\tools\rclone-v1.60.1-windows-amd64\rclone.exe" "ls" "LLOD:~dir2" "-vv"]
2022/12/13 21:22:24 DEBUG : Creating backend with remote "LLOD:~dir2"
2022/12/13 21:22:24 DEBUG : Using config file from "C:\Users\RM\AppData\Roaming\rclone\rclone.conf"
2022/12/13 21:22:25 ERROR : : error listing: directory not found
2022/12/13 21:22:25 DEBUG : 3 go routines active
2022/12/13 21:22:25 Failed to ls with 2 errors: last error was: directory not found

...

C:\ProgramData\chocolatey\bin>rclone lsd LLOD:~dir2 -vv
2022/12/13 21:28:26 DEBUG : rclone: Version "v1.60.1" starting with parameters ["C:\ProgramData\chocolatey\lib\rclone.portable\tools\rclone-v1.60.1-windows-amd64\rclone.exe" "lsd" "LLOD:~dir2" "-vv"]
2022/12/13 21:28:26 DEBUG : Creating backend with remote "LLOD:~dir2"
2022/12/13 21:28:26 DEBUG : Using config file from "C:\Users\RM\AppData\Roaming\rclone\rclone.conf"
2022/12/13 21:28:27 ERROR : : error listing: directory not found
2022/12/13 21:28:27 DEBUG : 3 go routines active
2022/12/13 21:28:27 Failed to lsd with 2 errors: last error was: directory not found

...

C:\ProgramData\chocolatey\bin>rclone ls LLOD:20180409 -vv
2022/12/13 21:26:26 DEBUG : rclone: Version "v1.60.1" starting with parameters ["C:\ProgramData\chocolatey\lib\rclone.portable\tools\rclone-v1.60.1-windows-amd64\rclone.exe" "ls" "LLOD:20180409" "-vv"]
2022/12/13 21:26:26 DEBUG : Creating backend with remote "LLOD:20180409"
2022/12/13 21:26:26 DEBUG : Using config file from "C:\Users\RM\AppData\Roaming\rclone\rclone.conf"
3743744 file1.mdb
1748992 file2.mdb
1748992 file3.mdb
2022/12/13 21:26:27 DEBUG : 3 go routines active

...

[Ole (Ole Frost)]

Hi Datamovr,

Seems like ~ isn't allowed in (user created) folder names in OneDrive Business according to this:

Characters that aren't allowed in file and folder names in OneDrive for work or school
~ " # % & * : < > ? / \ { }.

(Leading and trailing spaces in file or folder names also aren't allowed.)

Source: Restrictions and limitations in OneDrive and SharePoint

I therefore guess the folders you see are some kind of (usually hidden) system folders.

Did you create them yourself? How?

[DataMovr]

Thanks so much for that link. I've been searching and reviewing many Onedrive and rclone special character articles and had not found one that specifically called out the ~ character.

It was a few years ago that I created those ~directories. I was able to create those directories using one of these methods:

• An rclone copy of a parent folder where ~directories are sub-directories
• Created the ~directories with a client like Cyberduck or Expandrive
• Using the Onedrive interface via web browser

[Ole (Ole Frost)]

Thanks, interesting situation.

I found this in the list a backend flags after reading this, this and this:

      --onedrive-encoding MultiEncoder               The encoding for the backend (default Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,LeftTilde,RightSpace,RightPeriod,InvalidUtf8,Dot)

Never tried it before, but suggest you first try this:

rclone lsd LLOD:~dir2 --onedrive-encoding None

and if it works, checks this more precise setting where I just removed LeftTilde:

rclone lsd LLOD:~dir2 --onedrive-encoding Slash,LtGt,DoubleQuote,Colon,Question,Asterisk,Pipe,BackSlash,Del,Ctl,LeftSpace,RightSpace,RightPeriod,InvalidUtf8,Dot

[DataMovr]

Ole,

Thank you again for the reply.

You seem to have solved this issue. Both options you posted worked and returned directories inside the parent folder ~dir2.

...
rclone lsd LLOD:~dir2 --onedrive-encoding None
-1 2017-05-04 13:20:22 3 Dir1
-1 2019-09-16 07:14:28 217 Dir2
-1 2020-06-18 11:45:30 0 Dir3

...

Next I tested with the purge command I was originally going for and I did not get the directory not found error. I have a separate issue I'm working with MS on dealing with a broken eDiscovery or preservation hold that did not remove cleanly.

So many thanks Ole. I appreciate the assistance. Happy Holidays

-- DataMovr

[Ole (Ole Frost)]

You are welcome, glad to hear, happy Holidays!