Rclone takes a long time to launch

Help and Support
1

What is the problem you are having with rclone?

Hi! since yesterday I find myself with a funny problem. Rclone takes a long time to launch and during that time it uses all my bandwidth. Once started I notice that I finally have several rclone services running, yet I actually only have one.

What is your rclone version (output from rclone version)

rclone v1.55.0

  • os/type: linux
  • os/arch: amd64
  • go/version: go1.16.2
  • go/linking: static
  • go/tags: cmount

Which OS you are using and how many bits (eg Windows 7, 64 bit)

ubuntu 20.04

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

Paste command here

rclone mount --allow-other --allow-non-empty --dir-cache-time 8760h --cache-dir=/mnt/eab0e7c5-12d4-44c2-b741-457f2314e840 --poll-interval 15s --vfs-cache-mode full --vfs-cache-max-size 3000G --vfs-read-ahead 256M --vfs-cache-max-age 8760h Gcrypt: /mnt/cryption --user-agent="ams"

2

capture 3
capture 31680×1050 397 KB

capture clone 4
capture clone 41293×306 69.1 KB

rclone capture 5
rclone capture 51661×900 164 KB

3

If you are enable the debug log, we can see what's going on.

4

I don't know why it does this, but I did a test by disabling the service at startup.
systemctl disable Gcrypt.service
then I restart the server and start the service manually.
systemctl start Gcrypt.service
From there everything seems correct so I reactivated the
systemctl enable Gcrypt.service
service then I restarted. This is what I get

rclone forum
rclone forum1672×278 46.7 KB

5

You need to add -vv to the startup and --log-file /tmp/rclone.log so we can get a log file as the systemd output doesn't give any information other than the service status.

6

Very well I will come back with the results when the problem returns :slight_smile:

thank you!

7

The problem would come from the .service which cannot kill the rclone sessions when I modify them or restart them manually because of the killmode = none
I will try this instead

[Unit]
Description=Gcrypt Daemon
After=syslog.target network.target

[Service]
User=jonathan
Group=jonathan
Type=notify
ExecStart=/usr/bin/rclone mount --allow-other --allow-non-empty --dir-cache-time 8760h --cache-dir=/mnt/eab0e7c5-12d4-44c2-b741-457f2314e840 --poll-interval 15s --vfs-cache-mode full ->
TimeoutStopSec=20
KillMode=mixed
SendSIGKill=yes
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

8

If you sigkill rclone, you get a mount point that won't function because IO is still stuck on it.

You want to stop all IO and have the proper require in your services so they stop and you can unmount the mount point with no IO/processing hitting it.

9

So the ideal would be for me to start the service myself when the server is restarted?

10

If you add the flags I asked to add in, start fresh and share the log, we can see what the issue with it mounting.

And for worse things:

That.

With

That

Means that it'll keep killing the process and over mounting with allow empty and create multiple rclone processing making things worse.

99% of the folks should never use allow non empty except for very, very specific use cases.

11

ok i will delete it then and try with only killmode = none

Thanks for the advice!

12

An example requires for me, would be my rclone is 'rclone.service' and any file that uses rclone has this in it's service file:

felix@gemini:/etc/systemd/system$ cat gmedia.service
[Unit]
Description=gmedia mergerfs mount
ConditionPathIsMountPoint=/local
Requires=rclone.service
After=rclone.service

[Service]
Type=forking
ExecStart=/usr/bin/mergerfs /cache/local:/GD /gmedia -o rw,use_ino,allow_other,func.getattr=newest,category.action=all,category.create=ff,cache.files=partial,dropcacheonclose=true
KillMode=process
Restart=on-failure

[Install]
WantedBy=multi-user.target

So if the required service goes down, it stops this service, which is exactly what I want.

13

Type=forking is important?

14

That's very specific for mergerfs as it forks a process.

For rclone, you want:

[Service]
Type=notify
15

ok i will try with KillMode=process

Thanks :slight_smile:

16

Sorry as I feel I'm confusing you more.

The purpose of that service file was to illustrate a 'required' line and nothing else.

My rclone service file is here:

I use KillMode=none as I don't want the OS killing it. I use required for all my services to ensure that no IO/processes are on the mount point and it unmounts.

17

So I only need to remove allow-non-empty and add --attr-timeout 5000h?

[Unit]
Description=Gcrypt Daemon
After=syslog.target network.target

[Service]
User=jonathan
Group=jonathan
Type=notify
ExecStart=/usr/bin/rclone mount --allow-other --dir-cache-time 5000h --attr-timeout 5000h --cache-dir=/mnt/eab0e7c5-12d4-44c2-b741-457f2314e840 --poll-interval 15s --vfs-cache-mode full --vfs-cache-max-size 3000G --vfs-read-ahead 256M --vfs-cache-max-age 5000h --vfs-read-chunk-size=64M --buffer-size=128M Gcrypt: /mnt/cryption --user-agent="ams"
TimeoutStopSec=20
KillMode=none
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target

18

It's not stupid I had precisely the "IO" problem in my opinion, because on occasion the server froze and resumed its activity 1 hour later or by forcing the restart.
Do you think --allow-non-empty could cause this problem?

19

I wouldn't say anything is stupid as each use case / situation tends to be unique.

It could, but I like to not guess and see log files as that leaves it for less debate.

I have all my settings documented based on my use and they work great for my setup.

I give some reasoning for changing each line so you can always adapt. If setting attr timeout fits for you, you can change. I advise to leave anything at a default unless I know why I'm changing it.

20

i read about --attr-timeout but I am not sure I fully understand his role as well as should be a fair value for him :roll_eyes: