Backup copy to AWS S3

What is the problem you are having with rclone?

I have configured our SQL Server to perform backups & maintenance over the weekend. The first job on the SQL Server is to take these backups across the (AWS) network to a backup server. The second job is to remotely execute rclone on the backup server to copy the backups over to S3. The problem is that if the job fails for whatever reason the SQL server just assumes the job completed successfully. What I need some help with is working out how rclone can report back to the calling SQL job (it's a powershell command in a SQL job : Invoke-Command -ComputerName backupsrvr01 -ScriptBlock {rclone copy c:\mount\vol1\......

What is your rclone version (output from rclone version)

1.53.4

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Windows Server 2019 Datacenter (AWS EC2 Instance)

Which cloud storage system are you using? (eg Google Drive)

AWS S3

The command you were trying to run (eg rclone copy /tmp remote:tmp)

As described above, this is being executed from a SQL Server job calling rclone via powershell

rclone copy c:\mount\vol1\dbsrvr1 remote:S3\vol1\dbsrvr1 --config=c:\scripts\rclone.conf --filter "+ *FULL*.bak" --filter "- *" --ignore-checksum --ignore-existing --progress --s3-use-accelerate-endpoint

The rclone config contents with secrets removed.

[remote]
type = s3
provider = AWS
env_auth = true
region = eu-west-2

A log from the command with the -vv flag

There's no error in the log, it just stops logging - for example - in this particular instance the backup server rebooted (part of it's scaling down after the weekend) - so the logging entry is just where it was last copying and the SQL Server job just assumes the job ended successfully.

Rclone should return a non zero exit code on failure. I know nothing, alas, about powershell or SQL jobs, but I would have thought you should be able to detect a non-zero exit code.

I found this on stack overflow which looks like it might help

Thanks Nick - that's inspired me to go and dig around invoke-command powershell and error return stuff! cheers

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.