Bash script with whitespaces in variable

Help and Support
1

What is the problem you are having with rclone?

Using Rclone in a bash script but all things i try to escape whitespaces fails.

What is your rclone version (output from rclone version)

rclone v1.55.1

  • os/type: linux
  • os/arch: amd64
  • go/version: go1.16.3
  • go/linking: static
  • go/tags: none

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Operating System: Ubuntu 18.04.5 LTS
Kernel: Linux 4.15.0-153-generic
Architecture: x86-64

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

if [[ $# -eq 1 ]]
then
    source_dir=${1@Q}
fi

{ time $rclone move $source_dir GoogleDriveCrypt:  --config ~/.config/rclone/rclone.conf 
--progress --checkers 10 --stats-log-level ERROR --log-level INFO 
--log-file ~/logs/rclone/uploadMove.log --stats 5m  --tpslimit 10 --transfers 5 
--drive-chunk-size 256M --max-backlog 20000 --bwlimit 100M --rc --rc-addr $ip_port 
--rc-web-gui --rc-web-gui-update --rc-user=$user --rc-pass=$pass 
--rc-web-gui-no-open-browser --dry-run; } >> ~/logs/rclone/task.log 2>&1

A log from the command with the -vv flag

Command move needs 2 arguments maximum: you provided 6 non flag arguments: ["'/mnt/local/my" "pictures" "from" "last" "holiday'" "GoogleDriveCrypt:"]

with

source_dir=${1@Q}

i get an string for rclone like this:
'/mnt/local/my pictures from last holiday'

What am i not thinking of?

2

hello,

whenever there is a file path with a space character, it must be enclosed with quotes....

not a linux expert but perhaps try
rclone move "$source_dir"

3

i already do this with a parameter expansion.

source_dir=${1@Q}

$1 is the first argument passed to the script
${1} is the same as $1
@Q is the parameter expansion described here (search for " ${parameter@operator}":
https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html

4

well, based on the rclone error, it is not getting a quoted string it can use.

what is the exact output of echo $source_dir

have you tried rclone move "$source_dir" and if it failed, what was the exact rclone error message?

5

echo $source_dir
'/mnt/local/my pictures from last holiday'

using ""

echo $source_dir
"/mnt/local/my pictures from last holiday"

rlcone log

Command move needs 2 arguments maximum: you provided 6 non flag arguments:
["\"/mnt/local/my" "pictures" "from" "last" "holiday\"" "GoogleDriveCrypt:"]

with rclone move "$source_dir" it's the same, but error code is no 3 -> directory not found

6

what about a simple bash command instead of rclone, such as ls $source_dir

best to use the simplest command possible, post the exact command and entire debug output
rclone lsd $source_dir -vv

7

ls is the same:

ls $source_dir

ls: cannot access '"/mnt/local/my pictures from last holiday': No such file or directory
ls: cannot access 'pictures': No such file or directory
ls: cannot access 'from': No such file or directory
ls: cannot access 'last': No such file or directory
ls: cannot access 'holiday"': No such file or directory

So it's a bash thing... but why?

8

sorry, not a bash expert but at least it is not rclone...

try
echo $1
and
ls $1

9

works:

source_dir="./test test"
echo "source is : "$source_dir
ls "$source_dir"

fail:

source_dir="./test test"
echo "source is : "$source_dir
ls $source_dir

Ok now i have the solution, many thanks asdffdsa.
My problem was, that i mixing up all possibilities to escape the whitespaces. At the end i double escaped all paths :unamused: Furthermore single quotes won't work.

source_dir is now the path only
and the rclone command use the source_dir with the double quotes, like "$source_dir"

10

good, we got it working, as it can be very confusing, single quotes, double quotes, backticks and so on...
and good as i am working on my bash skills and in my testing, using double quotes with variable expansion keeps rclone happy...

11

Singe quotes return string literal. So '$foo' returns the literal string $foo and doesn't expand the variable.

You need to double quote variables if you want them to be passed a single value to a program. We can see this pretty easily with a test program:

#!/bin/sh
echo 1=$1 2=$2

Now if we call this with and without quotes

$ foo="hello there"
$ ./x $foo 
1=hello 2=there
$ ./x "$foo"
1=hello there 2=

There's other fun impacts of quotes as well:

$ foo="hello     there"
$ echo $foo
hello there
$ echo "$foo"
hello     there

So, pretty much anywhere a variable needs to be treated as a single value and not interpreted then surround it with "

12

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.