Rclone and cron

Trevor_Miles · March 10, 2022, 11:42am

What is the problem you are having with rclone?

When I run it from a bash script it does not seem to be able to pick up the necessary config information from /root/.config/rclone/rclone.conf. This produces an error that rclone is running using default values.

Run the command 'rclone version' and share the full output of the command.

clone v1.57.0
- os/version: raspbian 11.2 (64 bit)
- os/kernel: 5.10.92-v8+ (aarch64)
- os/type: linux
- os/arch: arm
- go/version: go1.17.2
- go/linking: static
- go/tags: none

Are you on the latest version of rclone? You can validate by checking the version listed here: Rclone downloads
--> Yes

Which cloud storage system are you using? (eg Google Drive)

OneDrive
GoogleDrive
GooglePhotos

They all report the same issue when running a bash shell.

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

The bash shell is:

#!/bin/bash
echo "$(date) - Synchronising OneDrive"
rclone sync OneDrive: /mnt/1TB_HDD/OneDrive -P -v --log-file=/home/pi/OneDrive.log
sudo chown pi /home/pi/OneDrive.log

echo "$(date) - Synchronizing GoogleDrive"
rclone sync GoogleDrive: /mnt/1TB_HDD/GoogleDrive -P -v --log-file=/home/pi/GoogleDrive.log
sudo chown pi /home/pi/GoogleDrive.log

echo "$(date) - Synchronizing GooglePhotos"
rclone sync GooglePhotos:media/all /mnt/1TB_HDD/GoogelPhotos -P -v --log-file=/home/pi/GooglePhotos.log
sudo chown pi /home/pi/GooglePhotos.log

echo "$(date) - Synchronising finished"

The crontab entry is

* 12 * * * /home/pi/autosync.sh > /home/pi/autosync.log 2 >> &1

The content of the latest autosync.log file is

Thu 10 Mar 06:00:01 GMT 2022 - Synchronising OneDrive
Thu 10 Mar 06:00:01 GMT 2022 - Synchronizing GoogleDrive
Thu 10 Mar 06:00:01 GMT 2022 - Synchronizing GooglePhotos
Thu 10 Mar 06:00:01 GMT 2022 - Synchronising finished

The rclone config contents with secrets removed.

[GoogleDrive]
type = drive
scope = drive
token = {"access_token":"XXXXXXXXXXXXXXXXXXXXXXXXXX"}
team_drive = 

[OneDrive]
type = onedrive
token = {"access_token":"XXXXXXXXXXXXXXXXXXXXXXX","token_type":"Bearer","refresh_token":"YYYYYYYYYYYYYYYYYYYYY","expiry":"2022-03-08T14:26:16.799799434Z"}
drive_id = 23a396028c0b7068
drive_type = personal

[GooglePhotos]
type = google photos
token = {"access_token":"XXXXXXXXXXXXXXXXXXXXXX","token_type":"Bearer","refresh_token":"YYYYYYYYYYYYYYYYYYYYYY","expiry":"2022-03-08T14:59:44.406217107Z"}

A log from the command with the `-vv` flag

OneDrive

2022/03/09 00:00:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/09 00:00:01 Failed to create file system for "OneDrive:": didn't find section in config file
2022/03/09 06:00:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/09 06:00:01 Failed to create file system for "OneDrive:": didn't find section in config file
2022/03/09 12:00:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/09 12:00:01 Failed to create file system for "OneDrive:": didn't find section in config file
2022/03/09 18:00:02 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/09 18:00:02 Failed to create file system for "OneDrive:": didn't find section in config file
2022/03/10 00:00:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/10 00:00:01 Failed to create file system for "OneDrive:": didn't find section in config file
2022/03/10 06:00:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/10 06:00:01 Failed to create file system for "OneDrive:": didn't find section in config file
2022/03/10 09:10:59 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/10 09:10:59 Failed to create file system for "OneDrive:": didn't find section in config file
2022/03/10 09:14:43 INFO  : HomeAssistant/config/config/configuration.yaml: Copied (replaced existing)
2022/03/10 09:14:43 INFO  : HomeAssistant/config/config/home-assistant.log.1: Copied (replaced existing)
2022/03/10 09:14:43 INFO  : HomeAssistant/config/config/secrets.yaml: Copied (replaced existing)
2022/03/10 09:14:44 INFO  : HomeAssistant/config/config/sensors.yaml: Copied (replaced existing)
2022/03/10 09:14:44 INFO  : HomeAssistant/config/config/automations.yaml: Copied (replaced existing)
2022/03/10 09:14:45 INFO  : HomeAssistant/config/config/.HA_VERSION: Copied (replaced existing)
2022/03/10 09:14:45 INFO  : HomeAssistant/config/config/daikin_puredata.json: Copied (replaced existing)
2022/03/10 09:14:46 INFO  : HomeAssistant/config/config/home-assistant_v2.db-shm: Copied (replaced existing)
2022/03/10 09:14:47 INFO  : HomeAssistant/config/config/zigbee.db: Copied (replaced existing)
2022/03/10 09:14:49 INFO  : HomeAssistant/config/config/.cloud/production_auth.json: Copied (replaced existing)

It then seems to sync the appropriate files.

GoogleDrive

2022/03/09 00:00:02 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/09 00:00:02 Failed to create file system for "GoogleDrive:": didn't find section in config file
2022/03/09 06:00:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/09 06:00:01 Failed to create file system for "GoogleDrive:": didn't find section in config file
2022/03/09 12:00:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/09 12:00:01 Failed to create file system for "GoogleDrive:": didn't find section in config file
2022/03/09 18:00:02 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/09 18:00:02 Failed to create file system for "GoogleDrive:": didn't find section in config file
2022/03/10 00:00:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/10 00:00:01 Failed to create file system for "GoogleDrive:": didn't find section in config file
2022/03/10 06:00:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/10 06:00:01 Failed to create file system for "GoogleDrive:": didn't find section in config file
2022/03/10 09:10:59 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/10 09:10:59 Failed to create file system for "GoogleDrive:": didn't find section in config file
2022/03/10 09:33:32 INFO  : Home Assistant Snapshots/Full Backup 2022-03-09 00:00:00.tar: Multi-thread Copied (new)
2022/03/10 09:34:13 INFO  : Home Assistant Snapshots/Full Backup 2022-03-10 00:00:00.tar: Multi-thread Copied (new)
2022/03/10 09:34:13 INFO  : Home Assistant Snapshots/core_2022.2.2.tar: Deleted
2022/03/10 09:34:13 INFO  : Home Assistant Snapshots/core_2022.3.2.tar: Deleted
2022/03/10 09:34:13 INFO  : 
Transferred:   	    6.497 GiB / 6.497 GiB, 100%, 21 B/s, ETA 0s
Checks:                17 / 17, 100%
Deleted:                2 (files), 0 (dirs)
Transferred:            2 / 2, 100%
Elapsed time:     13m40.9s

GooglePhotos

2022/03/09 00:00:02 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/09 00:00:02 Failed to create file system for "GooglePhotos:media/all": didn't find section in config file
2022/03/09 06:00:02 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/09 06:00:02 Failed to create file system for "GooglePhotos:media/all": didn't find section in config file
2022/03/09 12:00:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/09 12:00:01 Failed to create file system for "GooglePhotos:media/all": didn't find section in config file
2022/03/09 18:00:02 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/09 18:00:02 Failed to create file system for "GooglePhotos:media/all": didn't find section in config file
2022/03/10 00:00:02 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/10 00:00:02 Failed to create file system for "GooglePhotos:media/all": didn't find section in config file
2022/03/10 06:00:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/10 06:00:01 Failed to create file system for "GooglePhotos:media/all": didn't find section in config file
2022/03/10 09:10:59 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
2022/03/10 09:10:59 Failed to create file system for "GooglePhotos:media/all": didn't find section in config file
2022/03/10 09:34:56 NOTICE: einstein {AK5W7F4KcF0fdM_c37gPM5GaTbKsHGIgV1qwC0X8gGNmN_Kb9MMlSGfky0QUs6uM5q4cIFmUljfSHqpN2M7zmhyV3nMpPRN3lQ}.jpg: Duplicate object found in source - ignoring
2022/03/10 09:34:56 INFO  : There was nothing to transfer
2022/03/10 09:34:56 INFO  : 
Transferred:   	          0 B / 0 B, -, 0 B/s, ETA -
Checks:              6457 / 6457, 100%
Elapsed time:        42.9s

asdffdsa · March 10, 2022, 1:43pm

by default, rclone looks for the config file in a subfolder of the current user.
at ~/.config/rclone/rclone.conf
the script is run as user pi
the cron job is run as root

so if you want to run rclone as root and the config file is at /home/pi/.config/rclone/rclone.conf
need to tell rclone where the config file is by adding
--config=/home/pi/.config/rclone/rclone.conf

and i think that rclone log contains the results of both,.
--- the output when rclone is run as user pi which does work.
--- the output when rclone is run as user root using cron which does not work.

Trevor_Miles · March 10, 2022, 4:11pm

I had already copied the rclone.conf to /root/.config/rclone

I will try your suggestion.

asdffdsa · March 10, 2022, 4:29pm

this command will locate the config file
https://rclone.org/commands/rclone_config_file/

as a test, might run that command with cron and check the output.

Pal_Toth · March 12, 2022, 6:27pm

In your bash file before rclone command export your configfile:

RCLONE_CONFIG=/FULL_ROUTE_TO_YOUR_CONFIG/rclone.conf
export RCLONE_CONFIG

system · April 11, 2022, 6:28pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.