Filtering copy job log results for systemd

What is the problem you are having with rclone?

I'm running an rclone copy job as a systemd service to backup data to an offsite location. My upload speed is slow, and I have tens of thousands of files added daily, so I'm trying to configure what rclone outputs to the systemd journal so that the log isn't massive. As far as I can tell --log-systemd is only used by mount and rcd, so I'm ignoring it.

What I'm trying to achieve is a one-line, total progress report every minute or so. I do not need to capture which files have been transferred. I have tried the obvious:

rclone copy SRC TGT --progress --stats 1m --stats-one-line

On the console, this prints exactly what I want in the format I want. For the service, however, the output doesn't flush to the systemd journal until rclone exits, so it's not useful for tracking actively running transfers. Additionally, all of the one-line progress updates are on a single, long line. From other work I've done, I know that this is because the --progress updates don't include a new line character at the end.

The closest I've come is the following in the log:

rclone copy SRC TGT -v --stats 1m --stats-one-line

This also prints the progress report I want, but it also includes INFO lines when every file successfully transfers. Given the number of files I'm transferring, that bloats the log pretty quickly. It's sort of a workable solution, but not ideal.

Otherwise, I haven't found anything else in the docs that quite achieves exactly what I want. Is there another configuration or more log filtering available that I'm missing?

Run the command 'rclone version' and share the full output of the command.

rclone v1.59.2
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-50-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.18.6
- go/linking: static
- go/tags: none

You probably want to remove the -v and add --stats-log-level NOTICE

That's perfect! I was seeing the progress line reporting as INFO when -v was enabled and I assumed that was its the only log level. Thanks!

1 Like

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.