I get issues if a folder contains files with spaces in the name

Help and Support
1

Hey, has anyone else had an issue where they can't sync because the folder contains files with spaces in the filenames?
I have no idea how to work around this one. I mean, I can't wrap quotes around it, because the problem arises when RClone gets the list of files to be synced...

rclone version

rclone v1.67.0
- os/version: endeavouros (64 bit)
- os/kernel: 6.6.39-1-lts (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.22.5
- go/linking: dynamic
- go/tags: none

Storage platform:

Backblaze B2

Command:

rclone sync 
    --metadata 
    --verbose
    --progress 
    --track-renames 
    --color ALWAYS 
    --b2-encoding "Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot,RightSpace,LeftSpace" 
    --human-readable 
  ~/Audiobooks bblaze:Media/Audiobooks;

Also tried it without verbose, progress, track-renames human-readable, and without RightSpace,LeftSpace added to b2-encoding.

Source content:

ls -R ~/Audiobooks/
/home/nomad/Audiobooks/:
'Count_of_Monte Cristo'/   .stfolder/

'/home/nomad/Audiobooks/Count_of_Monte Cristo':
001-010/  020-030/  040-050/  060-070/  080-090/  100-110/  Count_of_Monte_Cristo_1308.jpg
010-020/  030-040/  050-060/  070-080/  090-100/  110-117/

'/home/nomad/Audiobooks/Count_of_Monte Cristo/001-010':
countofmontecristo_001_dumas_64kb.mp3  countofmontecristo_005_dumas_64kb.mp3  countofmontecristo_009_dumas_64kb.mp3
countofmontecristo_002_dumas_64kb.mp3  countofmontecristo_006_dumas_64kb.mp3  countofmontecristo_010_dumas_64kb.mp3
countofmontecristo_003_dumas_64kb.mp3  countofmontecristo_007_dumas_64kb.mp3
countofmontecristo_004_dumas_64kb.mp3  countofmontecristo_008_dumas_64kb.mp3

'/home/nomad/Audiobooks/Count_of_Monte Cristo/010-020':
countofmontecristo_011_dumas_64kb.mp3  countofmontecristo_015_dumas_64kb.mp3  countofmontecristo_019_dumas_64kb.mp3
countofmontecristo_012_dumas_64kb.mp3  countofmontecristo_016_dumas_64kb.mp3  countofmontecristo_020_dumas_64kb.mp3
countofmontecristo_013_dumas_64kb.mp3  countofmontecristo_017_dumas_64kb.mp3
countofmontecristo_014_dumas_64kb.mp3  countofmontecristo_018_dumas_64kb.mp3
# yadda yadda you get the idea.

// I tried renaming it, worked, changed it back, broke. It's definitely the space in the file path.

rclone config:

[bblaze]
type = b2
account = XXX
key = XXX
lifecycle = 30

Verbose log:

[nomad][~] $ rclone sync -MvvP --track-renames --color ALWAYS --b2-encoding "Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot,RightSpace,LeftSpace" --human-readable ~/Audiobooks bblaze:Media/Audiobooks;

2024/07/15 13:20:42 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "sync" "-MvvP" "--track-renames" "--color" "ALWAYS" "--b2-encoding" "Slash,BackSlash,Del,Ctl,InvalidUtf8,Dot,RightSpace,LeftSpace" "--human-readable" "/home/nomad/Audiobooks" "bblaze:Media/Audiobooks"]
2024/07/15 13:20:42 DEBUG : Creating backend with remote "/home/nomad/Audiobooks"
2024/07/15 13:20:42 DEBUG : Using config file from "/home/nomad/.config/rclone/rclone.conf"
2024/07/15 13:20:42 DEBUG : Creating backend with remote "bblaze:Media/Audiobooks"
2024/07/15 13:20:42 DEBUG : bblaze: detected overridden config - adding "{4NwYq}" suffix to name
2024/07/15 13:20:43 DEBUG : fs cache: renaming cache item "bblaze:Media/Audiobooks" to be canonical "bblaze{4NwYq}:Media/Audiobooks"
2024/07/15 13:20:43 INFO  : B2 bucket Media path Audiobooks: Making map for --track-renames
2024/07/15 13:20:43 INFO  : B2 bucket Media path Audiobooks: Finished making map for --track-renames
2024/07/15 13:20:43 DEBUG : B2 bucket Media path Audiobooks: Waiting for checks to finish
2024/07/15 13:20:43 DEBUG : B2 bucket Media path Audiobooks: Waiting for renames to finish
2024/07/15 13:20:43 DEBUG : Count_of_Monte Cristo/Count_of_Monte_Cristo_1308.jpg: Need to transfer - No matching file found at Destination
2024/07/15 13:20:43 ERROR : Count_of_Monte Cristo/Count_of_Monte_Cristo_1308.jpg: Failed to copy: failed to create bucket: bucketName must be at least 6 characters long (400 bad_request)
2024/07/15 13:20:43 DEBUG : Count_of_Monte Cristo/070-080/countofmontecristo_071_dumas_64kb.mp3: Need to transfer - No matching file found at Destination
2024/07/15 13:20:43 DEBUG : Count_of_Monte Cristo/070-080/countofmontecristo_076_dumas_64kb.mp3: Need to transfer - No matching file found at Destination
2024/07/15 13:20:43 ERROR : Count_of_Monte Cristo/070-080/countofmontecristo_071_dumas_64kb.mp3: Failed to copy: failed to create bucket: bucketName must be at least 6 characters long (400 bad_request)
2024/07/15 13:20:43 DEBUG : Count_of_Monte Cristo/070-080/countofmontecristo_072_dumas_64kb.mp3: Need to transfer - No matching file found at Destination
2024/07/15 13:20:43 DEBUG : Count_of_Monte Cristo/070-080/countofmontecristo_074_dumas_64kb.mp3: Need to transfer - No matching file found at Destination
2024/07/15 13:20:43 DEBUG : Count_of_Monte Cristo/070-080/countofmontecristo_075_dumas_64kb.mp3: Need to transfer - No matching file found at Destination
2024/07/15 13:20:43 ERROR : Count_of_Monte Cristo/070-080/countofmontecristo_076_dumas_64kb.mp3: Failed to copy: failed to create bucket: bucketName must be at least 6 characters long (400 bad_request)
2024/07/15 13:20:43 ERROR : Count_of_Monte Cristo/070-080/countofmontecristo_072_dumas_64kb.mp3: Failed to copy: failed to create bucket: bucketName must be at least 6 characters long (400 bad_request)
2024/07/15 13:20:43 DEBUG : Count_of_Monte Cristo/070-080/countofmontecristo_077_dumas_64kb.mp3: Need to transfer - No matching file found at Destination
# ... repetition redacted. 
2024/07/15 13:20:44 DEBUG : Count_of_Monte Cristo/001-010/countofmontecristo_006_dumas_64kb.mp3: Need to transfer - No matching file found at Destination
Transferred:   	         0 B / 209.795 MiB, 0%, 0 B/s, ETA -
Errors:                13 (retrying may help)
Transferred:            0 / 16, 0%
Elapsed time:         1.4s
Transferring:
 * Count_of_Monte Cristo/…sto_041_dumas_64kb.mp3:  0% /15.251Mi, 0/s, -
 * Count_of_Monte Cristo/…sto_048_dumas_64kb.mp3:  0% /15.082Mi, 0/s, -
 * Count_of_Monte Cristo/…sto_049_dumas_64kb.mp3:  0% /6.170Mi, 0/s, -
 * Count_of_Monte Cristo/…sto_050_dumas_64kb.mp3:  0% /11.499Mi, 0/s, -^C^C^C^C

OH by the way it does work as expected if I try synching a different folder that does NOT contain files with spaces in the title.

2

And, until I get this figured out, do you guys think it's safe to just let the backup run and get everything that would work right?
I mean, the real problem is that somewhere in there it's parsing the file names that have spaces as multiple arguments, which are all going to be invalid arguments because none of those one-word files exist...?

sync ~/Foo/Bar remote:Foo;
uploading local:~/Foo/Bar/Fizz Buzz.txt to remote:Foo.
Error: source file '~/Foo/Bar/Fizz' does not exist.
Error: destination folder 'Buzz.txt' does not exist.
3

Something fishy is going on here... I have tones of both folders and files with spaces synced without any issue. For years.

Maybe it is b2 specific?

Your last example - I have just tried with other remotes (not b2) and all works. Is there anybody else with b2 here able to test?

4

I wonder if it is... Looks to me like something should've been wrapped in quotes or escaped, between getting the list of files and assembling the upload command for the remote server's API.

5

welcome to the forum,

fwiw, start with the simplest command possible, use defaults, no flags.
then add flags one-by-one until the issue occurs.

rclone sync d:\files\zork b2:zorkzork -vv 
DEBUG : rclone: Version "v1.67.0" starting with parameters ["c:\\data\\rclone\\rclone.exe" "sync" "d:\\files\\zork" "b2:zorkzork" "-vv"]
DEBUG : Creating backend with remote "d:\\files\\zork"
DEBUG : Using config file from "c:\\data\\rclone\\rclone.conf"
DEBUG : fs cache: renaming cache item "d:\\files\\zork" to be canonical "//?/d:/files/zork"
DEBUG : Creating backend with remote "b2:zorkzork"
DEBUG :  f i l e . e x t: Need to transfer - File not found at Destination
DEBUG : file.ext: Need to transfer - File not found at Destination
DEBUG : B2 bucket zorkzork: Waiting for checks to finish
DEBUG : B2 bucket zorkzork: Waiting for transfers to finish
DEBUG : file.ext: sha1 = dcc22559fb8f17a0d37126678e02a363d566a57e OK
INFO  : file.ext: Copied (new)
DEBUG :  f i l e . e x t: sha1 = dcc22559fb8f17a0d37126678e02a363d566a57e OK
INFO  :  f i l e . e x t: Copied (new)
1 Like
6

I do not have b2 account to test. Anything else I tried works.

Can you try with some other remote?

7

Same issue even with just verbose-mode. rclone sync -vv ~/Audiobooks bblaze:Media/Audiobooks;, same reply.

Working on trying Google Drive or OneDrive

8

what is that? did not see the remote: in the config you posted???

why do you have the trailing semicolon in your commands. please test without it, ok?
the simplest command possible, ok?

i just demonstrated that is not a problem to sync from local to b2.
unless you think my test is not valid?

9

That was me hypothesizing about what's going on under the hood... All the real information is in the first post.
The command I was using is rclone sync ~/Audiobooks bblaze:Media/Audiobooks.
And the problem is that if there's files with spaces in the names, contained in the source folder, it fails.

ls ~/Audiobooks
'Count of Monte Cristo'/
10 
 rclone tree ~/Audiobooks
/
└── Count_of_Monte Cristo
    └── Count_of_Monte Cristo.txt

rclone copy ~/Audiobooks/ bblaze:zorkzork/Media/Audiobooks -vv
DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "copy" "/home/user01/Audiobooks/" "bblaze:zorkzork/Media/Audiobooks" "-vv"]                                                                                                                                                    2024/07/15 17:04:16 DEBUG : Creating backend with remote "/home/user01/Audiobooks/"
DEBUG : Using config file from "/home/user01/.config/rclone/rclone.conf"
DEBUG : fs cache: renaming cache item "/home/user01/Audiobooks/" to be canonical "/home/user01/Audiobooks"
DEBUG : Creating backend with remote "bblaze:zorkzork/Media/Audiobooks"                 
DEBUG : Count_of_Monte Cristo/Count_of_Monte Cristo.txt: Need to transfer - File not found at Destination
DEBUG : B2 bucket zorkzork path Media/Audiobooks: Waiting for checks to finish
DEBUG : B2 bucket zorkzork path Media/Audiobooks: Waiting for transfers to finish
DEBUG : Count_of_Monte Cristo/Count_of_Monte Cristo.txt: sha1 = da39a3ee5e6b4b0d3255bfef95601890afd80709 OK
INFO  : Count_of_Monte Cristo/Count_of_Monte Cristo.txt: Copied (new)
11

Tried it in a different context, still on B2. Still not working.

[nomad > test]$ ls -R ~/test/
/home/nomad/test/:
'test 1'/  'test 2'/

'/home/nomad/test/test 1':
'Testing 1.txt'

'/home/nomad/test/test 2':
'Testing 2.txt'

[nomad > test]$ alias rclone='rclone -vv'
[nomad > test]$ rclone mkdir bblaze:"Lets try this"
2024/07/15 15:33:27 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "-vv" "mkdir" "bblaze:Lets try this"]
2024/07/15 15:33:27 DEBUG : Creating backend with remote "bblaze:Lets try this"
2024/07/15 15:33:27 DEBUG : Using config file from "/home/nomad/.config/rclone/rclone.conf"
2024/07/15 15:33:27 DEBUG : B2 bucket Lets try this: Making directory
2024/07/15 15:33:28 ERROR : Attempt 1/3 failed with 1 errors and: failed to create bucket: Invalid characters in bucketName: must be alphanumeric or '-' (400 bad_request)
2024/07/15 15:33:28 DEBUG : B2 bucket Lets try this: Making directory
2024/07/15 15:33:28 ERROR : Attempt 2/3 failed with 1 errors and: failed to create bucket: Invalid characters in bucketName: must be alphanumeric or '-' (400 bad_request)
2024/07/15 15:33:28 DEBUG : B2 bucket Lets try this: Making directory
2024/07/15 15:33:28 ERROR : Attempt 3/3 failed with 1 errors and: failed to create bucket: Invalid characters in bucketName: must be alphanumeric or '-' (400 bad_request)
2024/07/15 15:33:28 DEBUG : 8 go routines active
2024/07/15 15:33:28 Failed to mkdir: failed to create bucket: Invalid characters in bucketName: must be alphanumeric or '-' (400 bad_request)

[nomad > test]$ rclone mkdir bblaze:"LetsTryThis"
2024/07/15 15:33:37 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "-vv" "mkdir" "bblaze:LetsTryThis"]
2024/07/15 15:33:37 DEBUG : Creating backend with remote "bblaze:LetsTryThis"
2024/07/15 15:33:37 DEBUG : Using config file from "/home/nomad/.config/rclone/rclone.conf"
2024/07/15 15:33:38 DEBUG : B2 bucket LetsTryThis: Making directory
2024/07/15 15:33:38 DEBUG : 8 go routines active

[nomad > test]$ rclone sync "~/test" bblaze:"LetsTryThis"
2024/07/15 15:35:28 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "-vv" "sync" "~/test" "bblaze:LetsTryThis"]
2024/07/15 15:35:28 DEBUG : Creating backend with remote "~/test"
2024/07/15 15:35:28 DEBUG : Using config file from "/home/nomad/.config/rclone/rclone.conf"
2024/07/15 15:35:28 DEBUG : fs cache: renaming cache item "~/test" to be canonical "/home/nomad/test/~/test"
2024/07/15 15:35:28 DEBUG : Creating backend with remote "bblaze:LetsTryThis"
2024/07/15 15:35:28 ERROR : Local file system at /home/nomad/test/~/test: error reading source root directory: directory not found
2024/07/15 15:35:28 DEBUG : B2 bucket LetsTryThis: Waiting for checks to finish
2024/07/15 15:35:28 DEBUG : B2 bucket LetsTryThis: Waiting for transfers to finish
2024/07/15 15:35:28 ERROR : B2 bucket LetsTryThis: not deleting files as there were IO errors
2024/07/15 15:35:28 ERROR : B2 bucket LetsTryThis: not deleting directories as there were IO errors
2024/07/15 15:35:28 ERROR : Attempt 1/3 failed with 1 errors and: directory not found
2024/07/15 15:35:28 ERROR : Local file system at /home/nomad/test/~/test: error reading source root directory: directory not found
2024/07/15 15:35:28 DEBUG : B2 bucket LetsTryThis: Waiting for checks to finish
2024/07/15 15:35:28 DEBUG : B2 bucket LetsTryThis: Waiting for transfers to finish
2024/07/15 15:35:28 ERROR : B2 bucket LetsTryThis: not deleting files as there were IO errors
2024/07/15 15:35:28 ERROR : B2 bucket LetsTryThis: not deleting directories as there were IO errors
2024/07/15 15:35:28 ERROR : Attempt 2/3 failed with 1 errors and: directory not found
2024/07/15 15:35:28 ERROR : Local file system at /home/nomad/test/~/test: error reading source root directory: directory not found
2024/07/15 15:35:28 DEBUG : B2 bucket LetsTryThis: Waiting for checks to finish
2024/07/15 15:35:28 DEBUG : B2 bucket LetsTryThis: Waiting for transfers to finish
2024/07/15 15:35:28 ERROR : B2 bucket LetsTryThis: not deleting files as there were IO errors
2024/07/15 15:35:28 ERROR : B2 bucket LetsTryThis: not deleting directories as there were IO errors
2024/07/15 15:35:28 ERROR : Attempt 3/3 failed with 1 errors and: directory not found
2024/07/15 15:35:28 INFO  : 
Transferred:   	         0 B / 0 B, -, 0 B/s, ETA -
Errors:                 1 (retrying may help)
Elapsed time:         0.6s

2024/07/15 15:35:28 DEBUG : 8 go routines active
2024/07/15 15:35:28 Failed to sync: directory not found

[nomad > ~]$ rclone lsf --max-depth 99 bblaze:
2024/07/15 15:41:51 DEBUG : rclone: Version "v1.67.0" starting with parameters ["rclone" "-Pvv" "lsf" "--max-depth" "99" "bblaze:"]
2024/07/15 15:41:51 DEBUG : Creating backend with remote "bblaze:"
2024/07/15 15:41:51 DEBUG : Using config file from "/home/nomad/.config/rclone/rclone.conf"
LetsTryThis/
Transferred:   	         0 B / 0 B, -, 0 B/s, ETA -
Elapsed time:         0.4s
2024/07/15 15:41:51 DEBUG : 8 go routines active
12

well, that is a completely different issue.

a bit confusing, some backends are bucket based. b2 is bucket based, s3 is bucket based.
whereas google drive and onedrive are not bucket based.

a bucket is not the same as a directory. https://www.backblaze.com/docs/cloud-storage-buckets

the rclone syntax for b2 remotes is remote:bucket/directory
so you tried to create a bucket named Lets try this with a space character. that is not allowed by b2.

13

My issue does seem to be exclusive to B2. I did manage to try it with OneDrive and got success.
Still hoping to figure it out with B2 as that's cheaper, but still.

14

I see. How about sub-directories though? "b2:LetsTryThis/testing testing" failed too.

15

sorry, very confusing, what is b2: ? did you create a new remote?

yes, check out my last example. the directory and the file, both had space character

16

Same remote, bblaze. Just indicating I was referring to any BackBlaze B2 Storage. Or really any remote with these "buckets" instead of root folders.

So, you're saying that it makes sense for a file like bblaze:test/testing testing.txt to be illegal? alright.

Happen to know of a utility that'll rename the files and just... idk, escape the spaces, replace them with underscores or dashes or just remove them? I don't mind if the names change, fixing symlinks won't be nearly as bad as learning this the hard way was. (think I only have like one or two, don't care about them)

17

sorry, no. but you can script it using bash, cmd, python, etc...

also, can mount the remote as local storage with rclone mount bblaze: ~/bblaze
and use whatever local tools you choose.

maybe @kapitainsky knows a linux utility

18

That's good to know. Thanks.
And nevermind on the file rename thing. I found something called Detox for this job.

I take back the nevermind, it's taking a while to figure out simply filtering out spaces

19

okay, now I'm really confused...
It worked finally, and I can't tell which difference was important.
There was some error about being unable to make whatever name I wanted (even if I did all lowercase and only letters), but when I tried a bucket name that was totally unique it starts working. Even with the spaces in the filenames.

It's working now, but I have no idea what I'm doing right that I was doing wrong before.

[nomad > ~]$ ls -lR ~/test
/home/nomad/test:
  - total 0
  - drwxr-xr-x 1 nomad nomad 26 Jul 15 15:20 'test 1'/
  - drwxr-xr-x 1 nomad nomad 26 Jul 15 19:55  test_2/
  - drwxr-xr-x 1 nomad nomad 26 Jul 15 19:56  test-3/
'/home/nomad/test/test 1':
  - total 4.0K
  - -rw-r--r-- 1 nomad nomad 2 Jul 15 15:14 'Testing 1.txt'
/home/nomad/test/test_2:
  - total 4.0K
  - -rw-r--r-- 1 nomad nomad 2 Jul 15 15:20 Testing_2.txt
/home/nomad/test/test-3:
  - total 0
  - -rw-r--r-- 1 nomad nomad 0 Jul 15 19:56 Testing-3.txt

[nomad > ~]$ rclone lsl bblaze:
        2 2024-07-15 15:14:19.331000000 nomadicvoxel-testing/test 1/Testing 1.txt
        0 2024-07-15 19:56:03.602000000 nomadicvoxel-testing/test-3/Testing-3.txt
        2 2024-07-15 15:20:24.951000000 nomadicvoxel-testing/test_2/Testing_2.txt
20

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.