Crontab not working on Mac (12.4)

Help and Support
1

What is the problem you are having with rclone?

I want an rclone backup task to run every day at 6pm. The cronjob I set up for it is however not working. I browsed the forum and the internet to learn how to work with crontab and managed to run test scripts, so the basics are working. I also gave "cron" permissions and made the sh.file executable with chmod +x. However, as soon as I add my rclone command to the cronjob or a shell script containing the rclone command, nothing executes.

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.0

  • os/version: darwin 12.4 (64 bit)
  • os/kernel: 21.5.0 (x86_64)
  • os/type: darwin
  • os/arch: amd64
  • go/version: go1.18.3
  • go/linking: dynamic
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

This is a the shell-script (Pidof was installed via homebrew):

#!/bin/bash

if pidof -o %PPID -x “Cronjob\ –\ All\ Volumes\ Unencrypted.sh”; then
exit 1
fi
rclone sync /Volumes/ jhdrive:/Backups/Unencrypted/ --transfers=5 --copy-links --create-empty-src-dirs --filter-from /Users/jan/Library/Mobile\ Documents/com\~apple\~CloudDocs/Rclone/Backups/Filter\ –\ All\ Volumes\ Unencrypted.txt --log-level INFO --log-file /Users/jan/Library/Mobile\ Documents/com\~apple\~CloudDocs/Rclone/Logs/Log\ –\ All\ Volumes\ Unencrypted.log --rc

And this is the crontab:

57 18 * * * /Users/jan/Library/Mobile\ Documents/com\~apple\~CloudDocs/Rclone/Backups/Cronjob\ –\ All\ Volumes\ Unencrypted.sh

Super curious what I am doing wrong here.

2

hi,

as a test,
--- remove the pidof detection
--- make sure to use full paths
--- run a simple command, such as rclone version
--- use a rclone debug log

1 Like
3

Hello, I tried to follow your suggestions. My sh-file looks like this now. I'm not sure how to incorporate the debug log though for the version command.

#!/bin/bash

if -o %PPID -x “/Users/jan/Library/Mobile\ Documents/com\~apple\~CloudDocs/Rclone/Backups/Cronjob\ –\ All\ Volumes\ Unencrypted.sh”; then
exit 1
fi
rclone version
exit
4

--- make sure the .sh file runs from the command line.
--- need to change the location of the log file
--- this is what i would test, again, without the pid stuff

#!/bin/bash

rclone version --log-level=DEBUG --log-file=/path/to/log.txt
5

thanks so much, this is what I executed now, however with no result of a log file appearing at the given destination:

crontab

* * * * * sudo /Users/jan/Library/Mobile\ Documents/com\~apple\~CloudDocs/Rclone/Backups/Cronjob\ –\ All\ Volumes\ Unencrypted.sh

command

#!/bin/bash

rclone version --log-level=DEBUG --log-file=/tmp/log.txt
6

did those other scripts run as root?
can you post an example?

i see that you are running the .sh as root.
in your example the cron tab entry contains sudo
not an expert, but not not sure you are doing it correctly.
https://mkyong.com/linux/linux-how-to-run-cron-job-as-root/

--- the formatting cron jobs entry, looks a bit strange/confusing, using space character without using enclosing quotes and both slashes, forward and backward?
--- perhaps keep the .sh in the /tmp dir
--- how did you create the cron entry; given that .sh script is to be run as root?
https://mkyong.com/linux/linux-how-to-run-cron-job-as-root/

--- use a very simple command

#!/bin/bash

echo test > /tmp/test.txt

--- make sure rclone can find and access the rclone.conf file.
maybe hardcode using --config=/path/to/rclone.conf

7

Thanks so much for the elaborate answer.

Exactly, the test scripts were exactly what you wrote down there (echo test...). And they worked. Even with the .sh in that directory.

Yes, I had sudo in the command, but that wasn't working either, so I removed it. Now it's just the time ***** plus the path to the script.

This is how I run rclone commands on Mac, I put the paths into Terminal first so I am sure they are correct and then implement these paths into the rclone command. When I run the rclone command purely in Terminal, everything works fine. Just in the cronjob scenario, things don't work anymore.

I created it with Mac Terminal with "crontab -e", then pressing "i" and then editing it.

In general I haven't found a working solution here not the forum nor on the web of how automating rclone tasks, like each day at a specific time for example.

I'd be happy for some resources/solutions of alternative ways if there are any.

Thanks again. Really appreciate it.

8

I am not 100% sure of your issues here but I've found a few things working with cron.

  1. I always put as much as possible into a script. So rather than having cron call a long command, I have cron call the script
  2. Always specify fully paths and assume your bash profiles aren't being loaded. You may be able to source them but I am not sure if that works
  3. Have lots of logging and debugging until it works. I've used set -x before but...
    • Use Python. Even if it can be a bash script, I've found that Python just makes it much easier. Even with rclone. So rather than
    #!/usr/bin/env bash
/full/path/to/rclone copy source: dest: --flags --other-flags --config /full/path/to/config
    I do
    #!/usr/bin/env python

import subprocess
subprocess.run(['/full/path/to/rclone', 'copy', 
    'source:', 'dest:', 
    '--flags', '--other-flags',
    '--config','/full/path/to/config',
])
    since it reduces the chances of me messing up bash escapes. It also makes it easier to comment and format the call.
  4. Test it when you can. I've had issues on macOS but on Linux, I've used something like this or this. Not sure what it would take to make it work on macOS

I hope that helps some

1 Like
9

Thanks so much Justin. This solution worked for me. I think you delivered two great blue-prints for automating rclone tasks on Mac to the community here.

10

to expand on @jwink3101 idea,
--- i have a python script that creates commands for rclone, 7zip, fastcopy, VSS.

about the function DoCmd, that runs the commands:
--- captures the stdout/stderr and logs that to text files.
---- if your rclone config file is crypted, need to pass that password to rclone.
that is handled by input

an example command is

RcloneCopyFilesCmd=c:\data\rclone\rclone.exe copy "b:\rclone\vss\en08_wasabi_veeam_br_en08_20220722.090710\BR_VSERVER03_EN08" "wasabi_vserver03brvserver03en08:en08/rclone/backup"   --immutable --stats=5h --fast-list --bwlimit=20M --include="*.{vib,vbk}" --max-age=90d --s3-no-check-bucket=true --s3-chunk-size=1G --bind=192.168.62.233  --ask-password --config="c:\data\rclone\rclone.conf" --log-level=DEBUG --log-file="c:\data\rclone\logs\en08_wasabi_veeam_br_en08\20220722.090710\rclone.log"

that command is passed to this function

def DoCmd(self, Text, Cmd, Input='', Env=''):
    try:
        if Common.DebugMode:
            Common.Log(self, f'DebugMode: DoCmd: {Text}={Cmd}')
            return
        Common.Log(self, f'DoCmd: Start:   {Text}={Cmd}')
        the_env = {};   the_env = dict(os.environ)
        if Env != '':   the_env.update(Env)
        x=subprocess.run(Cmd, cwd=Common.ScriptsDir, capture_output=True, text=True, input=Input, env=the_env)
        if len(x.stdout) != 0:  xfile = open(f'{Common.BackupLogDir}\\{Text}.stdout.log', "w");   xfile.write(x.stdout); xfile.close()
        if len(x.stderr) != 0:  xfile = open(f'{Common.BackupLogDir}\\{Text}.stderr.log', "w");   xfile.write(x.stderr); xfile.close()
        if x.returncode==0:
            print(x.stdout)
            Common.Log(self, f'DoCmd: End:   {Text}={Cmd}')
            return x
        else:
            Common.DoEnd(self, f'ERROR: DoCmd: End {Text}={Cmd} with returncode={x.returncode}')
    except Exception as exception:
        Common.DoEnd(self, f'ERROR: DoCmd={Cmd} - {traceback.format_exc()}')
1 Like
11

that looks very interesting, thanks looking into it. Hopefully useful for others on this forum too!

12

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.