rclone spamming syslog messages

robr · January 20, 2022, 2:24pm

Hi everyone,

first let me thank you for your contribution to this awesome project and looking at my issue. I want to be precise as possible.

TL;DR:

Installed rclone from AUR packages (yay -S rclone) on archlinux x86_64
Tried rclone with GoogleDrive and stopped the mounting, deleted the config
Syslog show massive load of log file (see sample below) with rclone help messages every second!
Even with rclone removed and no process running syslog is still full of it
I deleted the unit file in /etc/systemd/system/rclone.service, reloaded daemon and reset failed

What is the problem you are having with rclone?

after install rclone v.1.57.0 I have massive load of rclone messages in my syslog.

What is your rclone version (output from `rclone version`)

2022/01/20 08:52:31 DEBUG : rclone: Version "v1.57.0" starting with parameters ["rclone" "-vv"]

➜ log which rclone
/usr/bin/rclone

Which OS you are using and how many bits (e.g. Windows 7, 64 bit)

uname -a
Linux T450 5.16.1-arch1-1 #1 SMP PREEMPT Sun, 16 Jan 2022 11:39:23 +0000 x86_64 GNU/Linux

➜ log cat /etc/os-release

NAME="Arch Linux"
PRETTY_NAME="Arch Linux"
ID=arch
BUILD_ID=rolling
ANSI_COLOR="38;2;23;147;209"
HOME_URL="https://archlinux.org/"
DOCUMENTATION_URL="https://wiki.archlinux.org/"
SUPPORT_URL="https://bbs.archlinux.org/"
BUG_REPORT_URL="https://bugs.archlinux.org/"
LOGO=archlinux-logo

Which cloud storage system are you using? (e.g. Google Drive)

Google Drive from the rclone config (without API code)

The command you were trying to run (e.g. `rclone copy /tmp remote:tmp`)

I stopped rclone and deleted rclone via yay -R rclone. Still I receive syslog messages

A log from the command with the `-vv` flag (e.g. output from `rclone -vv copy /tmp remote:tmp`)

sudo syslog-ng -Fdev

[2022-01-20T08:58:36.726730] Outgoing message; message='Jan 10 07:46:18 T450 rclone[3993120]: --no-checksum Don\'t compare checksums on up/download\x0a' [2022-01-20T08:58:36.726746] Outgoing message; message='Jan 10 07:46:18 T450 rclone[3993120]: --no-modtime Don\'t read/write the modification time (can speed things up)\x0a' [2022-01-20T08:58:36.726761] Outgoing message; message='Jan 10 07:46:18 T450 rclone[3993120]: --no-seek Don\'t allow seeking in files\x0a' [2022-01-20T08:58:36.726802] Incoming log entry from journal; message=' --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true)' [2022-01-20T08:58:36.726819] json-parser(): no marker at the beginning of the message, skipping JSON parsing ; input=' --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true)', marker='@cee:' [2022-01-20T08:58:36.726829] json-parser(): no marker at the beginning of the message, skipping JSON parsing ; input=' --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true)', marker='@cim:' [2022-01-20T08:58:36.726870] Incoming log entry from journal; message=' --noapplexattr Ignore all "com.apple.*" extended attributes (supported on OSX only)' [2022-01-20T08:58:36.726885] json-parser(): no marker at the beginning of the message, skipping JSON parsing ; input=' --noapplexattr Ignore all "com.apple.*" extended attributes (supported on OSX only)', marker='@cee:' [2022-01-20T08:58:36.726893] json-parser(): no marker at the beginning of the message, skipping JSON parsing ; input=' --noapplexattr Ignore all "com.apple.*" extended attributes (supported on OSX only)', marker='@cim:' [2022-01-20T08:58:36.753231] Outgoing message; message='Jan 10 07:46:18 T450 rclone[3993120]: --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true)\x0a' [2022-01-20T08:58:36.753272] Outgoing message; message='Jan 10 07:46:18 T450 rclone[3993120]: --noapplexattr Ignore all "com.apple.*" extended attributes (supported on OSX only)\x0a' [2022-01-20T08:58:36.753303] Outgoing message; message='Jan 10 07:46:18 T450 rclone[3993120]: --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true)\x0a' [2022-01-20T08:58:36.753320] Outgoing message; message='Jan 10 07:46:18 T450 rclone[3993120]: --noapplexattr Ignore all "com.apple.*" extended attributes (supported on OSX only)\x0a' [2022-01-20T08:58:36.753361] Outgoing message; message='Jan 10 07:46:18 T450 rclone[3993120]: --noappledouble Ignore Apple Double (._) and .DS_Store files (supported on OSX only) (default true)\x0a' [2022-01-20T08:58:36.753382] Outgoing message; message='Jan 10 07:46:18 T450 rclone[3993120]: --noapplexattr Ignore all "com.apple.*" extended attributes (supported on OSX only)\x0a' [2022-01-20T08:58:36.753905] Running application hooks; hook='4'

THANK YOU very much for every hint how to fix this in advance.

Animosity022 · January 20, 2022, 2:32pm

You just need to kill the rclone process that's running or reboot your system.

Programs only do what you tell them. Based on your log, you asked for debug output (-vv) so that's what you got.

robr · January 20, 2022, 3:17pm

Hi @Animosity022 and thanks for taking your time.

That is understandable. But I removed und purged rclone. And still this logs do appear. I searched the entire system in the unit file and in all possible file contents with grep but cannot find rclone commands nor binaries.

What the heck is that? Thank you very much for your ideas what can cause this.

Animosity022 · January 20, 2022, 3:20pm

Not a file, but a process.

ps -ef | grep rclone

See if it's running or just reboot the server. Doesn't matter which you do.

robr · January 20, 2022, 3:32pm

thanks! I tried to reboot but without luck.

See the process is not running:

➜  ~ sudo ps -ef | grep rclone

[sudo] password for plex:
plex       50281   50151  0 16:31 pts/0    00:00:00 grep --color=auto --exclude-dir=.bzr --exclude-dir=CVS --exclude-dir=.git --exclude-dir=.hg --exclude-dir=.svn --exclude-dir=.idea --exclude-dir=.tox rclone

What is going on here? Thank you very much for your consideration and understanding my desperation

Animosity022 · January 20, 2022, 3:38pm

That means no rclone is running so no idea.

Did you reboot your system? I've asked that a few times now.

ncw · January 20, 2022, 4:11pm

Does the number here change? if not kill 3993120 should kill the rclone instance.

The text looks like the help of rclone so I suspect you've got an rclone on the crontab executing continuously.

robr · January 20, 2022, 4:26pm

that was a good idea! But unfortunately no entries in sudo crontab NOR crontab.

Additional the ID changes. It is now looking like this:

[2022-01-20T17:24:44.510183] Outgoing message; message='Jan 10 15:01:48 T450 rclone[4163780]:       --daemon-wait duration                   Time to wait for ready mount from daemon (maximum time on Linux, constant sleep time on OSX/BSD) (not supported on Windows) (default 1m0s)\x0a'
[2022-01-20T17:24:44.510210] Outgoing message; message='Jan 10 15:01:48 T450 rclone[4163780]:       --daemon                                 Run mount in background and exit parent process (as background output is suppressed, use --log-file with --log-format=pid,... to monitor) (not supported on Windows)\x0a'
[2022-01-20T17:24:44.510232] Outgoing message; message='Jan 10 15:01:48 T450 rclone[4163780]:       --daemon-timeout duration                Time limit for rclone to respond to kernel (not supported on Windows)\x0a'
[2022-01-20T17:24:44.510249] Outgoing message; message='Jan 10 15:01:48 T450 rclone[4163780]:       --daemon-wait duration                   Time to wait for ready mount from daemon (maximum time on Linux, constant sleep time on OSX/BSD) (not supported on Windows) (default 1m0s)\x0a'
[2022-01-20T17:24:44.510823] Running application hooks; hook='4'

Animosity022 · January 20, 2022, 4:27pm

Did you reboot your system?

robr · January 20, 2022, 4:29pm

yes I rebooted the system multiple times since.

Animosity022 · January 20, 2022, 4:31pm

So if you've removed rclone, purged it, rebooted, checked for the actual binary, something doesn't make sense.

Binaries don't run magically.

How are you checking for existence of the binary? There really isn't anything 'rclone' here as this is just sys admin support at this point.

ncw · January 20, 2022, 4:34pm

It looks like there is an rclone configured in systemd. The configuration is wrong hence the --help message and maybe it is retrying it again and again?

I see that you attempted to delete the unit file and restart reload the daemon - it looks like that failed.

robr · January 20, 2022, 4:34pm

@Animosity022 I agree that is why I am desparate here.

Last call could be that my syslog is showing bogus as sudo dmesg is not showing rclone instances. Only the syslog command is showing this with:

sudo syslog-ng -Fdev

Could this be a trace?

Animosity022 · January 20, 2022, 4:38pm

If there isn't a binary file and the system was rebooted, you wouldn't have running rclone since there isn't a binary to spawn and it would error out / fail after a few tries.

As root

 find / -name *rclone*

If there is no binary, nor systemd unit files, nor any crontabs, you have a possessed server.

ncw · January 20, 2022, 4:40pm

Grepping for rclone in /etc/systemd is probably a good idea

grep -R rclone /etc/systemd

robr · January 20, 2022, 4:43pm

that is really interesting. If I execute this the following output it get:

➜  ~ grep -R rclone /etc/systemd

grep: /etc/systemd/system/multi-user.target.wants/rclone.service: No such file or directory

It looks like a phantom unit file the system is missing? How to repair this?

ncw · January 20, 2022, 4:55pm

Re-installing the package will probably fix it.

I suggest you do that then use systemctl disable to disable the rclone service.

Note that we (the rclone team) don't make the Arch packages, and I personally don't use Arch so you might be better asking on an Arch forum if that doesn't work!

Animosity022 · January 20, 2022, 5:15pm

That's nothing as that's a symbolic link for systemd as you had a requirement in your .service file.

If you look, you can just remove the broken link, no harm no foul.

You'll see it there:

/etc/systemd/system/multi-user.target.wants

as a broken rclone.service link.

robr · January 20, 2022, 6:30pm

I tried this and also reinstalled rclone. After serveral reboots the systems still shows the logs. I notice that my systems is also freezing after here and there. Maybe @Animosity022 is right about a compromised system? I am confused.

system · February 19, 2022, 6:31pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.