hi guys
i am using a script to create daily backups of a folder + sql file on a server running centos using rclone that uploads to azure.
this script works fine when I run it via ssh console, but if I add it as cron job in /etc/crontab, the script runs, creates the dump.sql, puts the archive in tmp folder, but does not upload it to azure. i am guessing it fails at the rclone part.
here is the script,what is the problem here, can someone tell me?
#!/usr/bin/env bash
set -e
set -u
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
export PATH
today=$(date +%d-%m-%Y-%H:%M)
logs="/rclone.log"
# name of the directory where the archive_file will be created
backupdir="/tmp"
# Container name on AZURE
container="apache-backup"
# name and location of sql dump file
sql_dump_file="/dump.sql"
# AZURE location
backend="azure:${container}"
# name of the archive file, change backup_ to backup_servername if using script on multiple different servers to avoid confusion and possibly overwriting of backups.
archive_file="$backupdir/backup_$(date '+%Y-%m-%d_%H%M%S').tar.gz"
# Rclone exports for connecting to AZURE
export RCLONE_CONFIG_AZURE_TYPE=azureblob
export RCLONE_CONFIG_AZURE_ACCOUNT=xxxxx
export RCLONE_CONFIG_AZURE_KEY=xxxxxxxx
# dumps the sql database into sql_dump_file
echo $today "Creating sql dump" >> $logs
mysqldump --quick --single-transaction --all-databases > "${sql_dump_file}"
if [ -e "${sql_dump_file}" ] ; then
echo $today "sql dump file created" >> $logs
else
echo $today "creation of sql dump failed, exiting" >> $logs
exit 1
fi
# archives the dirs var/www, root and the sql_dump_file into a tar.gz file (excluding .gnupg dir,
echo $today "Creating archive file" >> $logs
/usr/bin/tar --exclude='.gnupg' -czf "${archive_file}" /var/www "${sql_dump_file}"
if [ -e "${archive_file}" ] ; then
echo $today "Archive file created" >> $logs
else
echo $today "Creation of Archive file failed, cleaning and exiting" >> $logs
rm -rf "${sql_dump_file}"
exit 1
fi
# copies the archive file to the AZURE backup location
echo $today "Copying archive to AZURE" >> $logs
/usr/bin/rclone copy -v "${archive_file}" "${backend}" >> $logs 2>&1
echo $today "Cleanup" >> $logs
echo "----------------------------------------------------" >> $logs
rm "${archive_file}" "${sql_dump_file}"
exit 0
The usual problem people have with rclone in crontabs is it not being able to find its config file, so you can try adding the --config /path/to/config directive. Use rclone config file to show where it is.
rclone config
2018/12/05 17:26:54 NOTICE: Config file “/root/.config/rclone/rclone.conf” not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
The script runs without a config file set because we mention variables in the script itself. Are you saying we should add a config for the remote destination?
the logs are empty.
If you see the script above, its actually doing so:
making sql dump
adding a folder to backup in archive + the sql dump made above
uploading file to rclone
deleting it locally
the error comes on part number 3 (uploading to rclone), if i run this script as cron.
this is because the .tar files keep accumulating in /tmp folder but are not uploaded to azure when I put on the cron.
if i run the same script via ssh terminal as ‘sh scriptfile.sh’, then there are no problems.
You can add a “-x” after the bash on the first line and that’ll dump all the output of the commands out when it runs. You can see which command is failing.
I think you are missing passing the config file location to rclone.