Problems with scripting rclone - unexpected behaviour

Help and Support
1

Hi,

I'm in the process of moving to B2 for my cloud storage and plan to run via a script rather than the FreeNAS GUI as there are some additional parameters I wish to use. However, I'm getting an error with my script and was wondering if anyone could help (currently testing on one dataset [my young daughters data] at the moment). I'm executing the script via a Putty Shell connection. https://www.shellcheck.net/ shows no error on my code.

#!/bin/bash

### Define Parameters
src=/mnt/tank/FamilyHannah
dest=b2encrypt:FamilyHannah
email=my@email.address
log_file=/tmp/rclonelog.txt
log_level=INFO
min_age=15m
transfers=16
###

### Execute
rclone sync \
    --transfers ${transfers} \
    --fast-list \
    --copy-links \
    --min-age ${min_age} \
    --log-level ${log_level} \
    --log-file ${log_file} \
    --b2-hard-delete \
    --exclude "Thumbs.db" \
    --exclude "desktop.ini" \
    --exclude "AlbumArt*" \
    --exclude ".recycle/**" \
    --exclude ".windows/**" \
    --dry-run \
    ${src} ${dest}
success=$?

if [[ $success != 0 ]]; then
    subject="rclone backup: An error occurred. Please check the logs. (error code:${success})"
    body="Refer to rclone for more information"
else
    subject="rclone backup: Backup succeeded"
    body=""
fi

### send the email using the email_attachments.sh script
/mnt/tank/Sysadmin/scripts/email_attachments.sh ${email} ${email} "${subject}" "${body}" /tmp/rclonelog.txt
###

### Tidy Up ###
## Delete the rclone log in preparation of a new log
rm /tmp/rclonelog.txt
### End ###

If I run the above script (using --dry-run) it works, and I get emailed the following:

2019/06/06 12:43:22 NOTICE: .windows: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Hannah school handwriting Book.docx: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Test.docx: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Untitled.png: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Hannahs House.png: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: jcob.png: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: adams house.png: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: our house.png: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: testing.txt: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: python/fun Q&A.py: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Favorites/BBC - CBeebies Home.url: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Favorites/Bing.url: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Favorites/Bob the Builder Official Site  Online Games, Videos, Activities, Printables.url: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Favorites/Disney Junior  New Official Disney Junior UK Website  Disney UK.url: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Northwood Primary School/Year 3/Half-term project on Volcanos.pub: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Favorites/NORAD Santa.url: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Favorites/Rabbits CCTV.url: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Northwood Primary School/Year 5/Grenada Hannah Year 5.pptx: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: LEGO Creations/Models/kjhgggtr5ewqasdxzcvb.lxf: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Favorites/Links/BBC - CBeebies - Home - Play fun games and early years activities.url: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Favorites/Links/Suggested Sites.url: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Favorites/Links/Disney Junior  New Official Disney Junior UK Website  Disney UK.url: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Favorites/Links/YouTube.url: Not copying as --dry-run
2019/06/06 12:43:22 NOTICE: Favorites/Links/Fun Educational Games for Kids  e-Learning Resources for Teachers  EducationCity UK.url: Not copying as --dry-run
2019/06/06 12:43:22 INFO  : Encrypted drive 'secret:FamilyHannah': Waiting for checks to finish
2019/06/06 12:43:22 INFO  : Encrypted drive 'secret:FamilyHannah': Waiting for transfers to finish
2019/06/06 12:43:22 INFO  : Waiting for deletions to finish
2019/06/06 12:43:22 INFO  :
Transferred:                0 / 0 Bytes, -, 0 Bytes/s, ETA -
Errors:                 0
Checks:                 0 / 0, -
Transferred:           24 / 24, 100%
Elapsed time:        2.4s

However, if I remove the "--dry-run " line, i get the following error on screen:

sage:
  rclone sync source:path dest:path [flags]

Flags:
  -h, --help   help for sync

Use "rclone [command] --help" for more information about a command.
Use "rclone help flags" for to see the global flags.
Use "rclone help backends" for a list of supported services.
Command sync needs 2 arguments minimum

However, if I extract the code and just run it as a CLI command, i.e.:

rclone sync --transfers 16 --fast-list --copy-links --min-age 15m --log-level INFO --log-file /tmp/rclonelog.txt --b2-hard-delete  --exclude "Thumbs.db" --exclude "desktop.ini" --exclude "AlbumArt*" --exclude ".recycle/**" --exclude ".windows/**"  /mnt/tank/FamilyHannah secret:FamilyHannah

It works and the log file says:

2019/06/06 12:53:20 INFO  : Encrypted drive 'secret:FamilyHannah': Waiting for checks to finish
2019/06/06 12:53:20 INFO  : Encrypted drive 'secret:FamilyHannah': Waiting for transfers to finish
2019/06/06 12:53:22 INFO  : .windows: Copied (new)
2019/06/06 12:53:22 INFO  : Test.docx: Copied (new)
2019/06/06 12:53:22 INFO  : Untitled.png: Copied (new)
2019/06/06 12:53:22 INFO  : Hannah school handwriting Book.docx: Copied (new)
2019/06/06 12:53:22 INFO  : Hannahs House.png: Copied (new)
2019/06/06 12:53:22 INFO  : jcob.png: Copied (new)
2019/06/06 12:53:22 INFO  : python/fun Q&A.py: Copied (new)
2019/06/06 12:53:22 INFO  : our house.png: Copied (new)
2019/06/06 12:53:22 INFO  : testing.txt: Copied (new)
2019/06/06 12:53:23 INFO  : Favorites/Rabbits CCTV.url: Copied (new)
2019/06/06 12:53:23 INFO  : Favorites/Links/Fun Educational Games for Kids  e-Learning Resources for Teachers  EducationCity UK.url: Copied (new)
2019/06/06 12:53:23 INFO  : Favorites/Links/YouTube.url: Copied (new)
2019/06/06 12:53:23 INFO  : Favorites/NORAD Santa.url: Copied (new)
2019/06/06 12:53:23 INFO  : Favorites/Disney Junior  New Official Disney Junior UK Website  Disney UK.url: Copied (new)
2019/06/06 12:53:23 INFO  : Favorites/BBC - CBeebies Home.url: Copied (new)
2019/06/06 12:53:23 INFO  : LEGO Creations/Models/kjhgggtr5ewqasdxzcvb.lxf: Copied (new)
2019/06/06 12:53:23 INFO  : Favorites/Links/BBC - CBeebies - Home - Play fun games and early years activities.url: Copied (new)
2019/06/06 12:53:23 INFO  : adams house.png: Copied (new)
2019/06/06 12:53:23 INFO  : Favorites/Links/Suggested Sites.url: Copied (new)
2019/06/06 12:53:23 INFO  : Favorites/Links/Disney Junior  New Official Disney Junior UK Website  Disney UK.url: Copied (new)
2019/06/06 12:53:24 INFO  : Favorites/Bob the Builder Official Site  Online Games, Videos, Activities, Printables.url: Copied (new)
2019/06/06 12:53:24 INFO  : Favorites/Bing.url: Copied (new)
2019/06/06 12:53:24 INFO  : Northwood Primary School/Year 3/Half-term project on Volcanos.pub: Copied (new)
2019/06/06 12:53:29 INFO  : Northwood Primary School/Year 5/Grenada Hannah Year 5.pptx: Copied (new)
2019/06/06 12:53:29 INFO  : Waiting for deletions to finish
2019/06/06 12:53:29 INFO  :
Transferred:        4.419M / 4.419 MBytes, 100%, 375.146 kBytes/s, ETA 0s
Errors:                 0
Checks:                 0 / 0, -
Transferred:           24 / 24, 100%
Elapsed time:         12s

So I'm not sure why the script is not working, any ideas?

Jonathan

2

Run your script with -vv. One of the first DEBUG logs will show exactly what the shell passed to rclone - that should give a clue.

3

I tried with --log-level DEBUG (instead of -vv - they the same I think), and got:

2019/06/08 21:56:48 DEBUG : --min-age 15m0s to 2019-06-08 21:41:48.97727359 +0100 BST m=-899.994073258
2019/06/08 21:56:48 DEBUG : rclone: Version "v1.44" starting with parameters ["rclone" "sync" "--transfers" "16" "--fast-list" "--copy-links" "--min-age" "15m" "--log-level" "DEBUG" "--log-file" "/tmp/rclonelog.txt" "--b2-hard-delete" "--exclude" "Thumbs.db" "--exclude" "desktop.ini" "--exclude" "AlbumArt*" "--exclude" ".recycle/**" "--exclude" ".windows"]
Usage:
  rclone sync source:path dest:path [flags]

Flags:
  -h, --help   help for sync

Use "rclone [command] --help" for more information about a command.
Use "rclone help flags" for to see the global flags.
Use "rclone help backends" for a list of supported services.
Command sync needs 2 arguments minimum

Looks like its not passing the ${src} ${dest}, will have to look into that more.

Jonathan

4

Right sorted it :slight_smile: Should have used DEBUG to start with.

Problem was that if I hash-out "#" the line "--dry-run ", because of the escape code at the end, it also hashed-out the ${src} ${dest} line... Just deleting "--dry-run " instead fixes it.

Thanks for the pointer.

Jonathan

2 Likes
5

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.