Need support with rclone service file

iragoiko · October 25, 2020, 2:42pm

Thanks mate! I was becoming crazy with this service file.
Now I can start service manually but after reboot it is down again with the follwoing log:

root@debian:/home/iragoiko/gdrive# sudo service rclone status -vv
● rclone.service - RClone Service
   Loaded: loaded (/etc/systemd/system/rclone.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sun 2020-10-25 15:38:23 CET; 1min 13s ago
  Process: 799 ExecStart=/usr/bin/rclone mount gdrive: /home/iragoiko/gdrive --config=/home/iragoiko/.config/rclone/rclone.conf --allow-other --dir-cache-time 96h --log-level INFO --log-file /home/iragoiko/rclone/rclone.log --poll-interval 15s --umask 002 --cache-dir=/cache --vfs-cache-mode full --vfs-cache
 Main PID: 799 (code=exited, status=1/FAILURE)

oct 25 15:38:23 debian systemd[1]: Starting RClone Service...
oct 25 15:38:23 debian systemd[1]: rclone.service: Main process exited, code=exited, status=1/FAILURE
oct 25 15:38:23 debian systemd[1]: rclone.service: Failed with result 'exit-code'.
oct 25 15:38:23 debian systemd[1]: Failed to start RClone Service.

asdffdsa · October 25, 2020, 2:49pm

you should look at the rclone config file.
try a simple command and seee what happens.
ExecStart=/usr/bin/rclone mount gdrive: /home/iragoiko/gdrive --config=/home/iragoiko/.config/rclone/rclone.conf --allow-other
you might need to add this to the [Service] section. change the user and group as needed.

User=root
Group=root

iragoiko · October 25, 2020, 3:14pm

1.-Rclone file:

[gdrive]
type = drive
client_id = 41496933802***********************************************gleuserconten$
client_secret = dMgo*********************NOlj4
scope = drive
token = {"access_token":"ya29.A0Af*******************************************************pfcK416u$

2.-When launching command " `ExecStart=/usr/bin/rclone mount gdrive: /home/iragoiko/gdrive --config=/home/iragoiko/.config/rclone/rclone.conf --allow-other"
mount: unrecognized option '--config=/home/iragoiko/.config/rclone/rclone.conf'
3.- I tried as root and as iragoiko (both user and group). No way.

I can run command sudo service rclone start -vv and it is working fine.
But systemctl commands are not working.

asdffdsa · October 25, 2020, 3:25pm

if you change the rclone.service file, then you need to post it, so we can see it.

iragoiko · October 25, 2020, 3:30pm

Sorry, see below rclone service file.

[Unit]
Description=RClone Service
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/iragoiko/.config/rclone/rclone.conf
ExecStart=/usr/bin/rclone mount gdrive: /home/iragoiko/gdrive \
--config=/home/iragoiko/.config/rclone/rclone.conf \
# This is for allowing users other than the user running rclone access to the mount
--allow-other \
# Google Drive is a polling remote so this value can be set very high and any changes are detected via polling.
--dir-cache-time 96h \
# The log level output
--log-level INFO \
# Location of the log file
--log-file /home/iragoiko/rclone/rclone.log \
# I reduce the poll interval down to 15 seconds as this makes changes appear fast the API quotas per day are huge
--poll-interval 15s \
# This is setting the file permission on the mount to user and group have the same access and other can read
--umask 002 \
# The local disk used for caching
--cache-dir=/cache \
# This is used for caching files to local disk for streaming
--vfs-cache-mode full \
# This limits the cache size to the value below
--vfs-cache-max-size 30G \
# This limits the age in the cache if the size is reached and it removes the oldest files first
--vfs-cache-max-age 336h \
# This sets up the remote control daemon so you can issue rc commands locally
--rc
ExecStop=/bin/fusermount -uz /home/iragoiko/gdrive
Restart=on-success
RestartSec=5
User=iragoiko
Group=iragoiko

[Install]
WantedBy=multi-user.target

asdffdsa · October 25, 2020, 3:43pm

change
ExecStart=/usr/bin/rclone mount gdrive: /home/iragoiko/gdrive \
to
ExecStart=/usr/bin/rclone mount gdrive: /home/iragoiko/gdrive

the trailing \ is when you have a multi-line command, but now you have commented out all the flags

iragoiko · October 25, 2020, 4:07pm

if I type as iragoiko:

sudo systemctl enable rclone.service
sudo systemctl start rclone.service
sudo systemctl status rclone.service

It says running and shows the gdrive content in the mount point. But when rebooting service is again down...
Current service file:

[Unit]
Description=RClone Service
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/iragoiko/.config/rclone/rclone.conf
ExecStart=/usr/bin/rclone mount gdrive: /home/iragoiko/gdrive
--config=/home/iragoiko/.config/rclone/rclone.conf \
# This is for allowing users other than the user running rclone access to the mount
--allow-other \
# Google Drive is a polling remote so this value can be set very high and any changes are detected via polling.
--dir-cache-time 96h \
# The log level output
--log-level INFO \
# Location of the log file
--log-file /home/iragoiko/rclone/rclone.log \
# I reduce the poll interval down to 15 seconds as this makes changes appear fast the API quotas per day are huge
--poll-interval 15s \
# This is setting the file permission on the mount to user and group have the same access and other can read
--umask 002 \
# The local disk used for caching
--cache-dir=/cache \
# This is used for caching files to local disk for streaming
--vfs-cache-mode full \
# This limits the cache size to the value below
--vfs-cache-max-size 30G \
# This limits the age in the cache if the size is reached and it removes the oldest files first
--vfs-cache-max-age 336h \
# This sets up the remote control daemon so you can issue rc commands locally
--rc
ExecStop=/bin/fusermount -uz /home/iragoiko/gdrive
Restart=on-success
RestartSec=5
User=iragoiko
Group=iragoiko

asdffdsa · October 25, 2020, 4:10pm

really, not a systemd expert.

perhaps add back the two flags for the rclone log and looks for errors there.

also, change
--log-level INFO
to
--log-level DEBUG

iragoiko · October 25, 2020, 4:25pm

Updated service file:

[Unit]
Description=RClone Service
Wants=network-online.target
After=network-online.target

[Service]
Type=notify
Environment=RCLONE_CONFIG=/home/iragoiko/.config/rclone/rclone.conf
ExecStart=/usr/bin/rclone mount gdrive: /home/iragoiko/gdrive
--config=/home/iragoiko/.config/rclone/rclone.conf \
# This is for allowing users other than the user running rclone access to the mount
--allow-other \
# Google Drive is a polling remote so this value can be set very high and any changes are detected via polling.
--dir-cache-time 96h \
# The log level output
--log-level DEBUG \
# Location of the log file
--log-file /home/iragoiko/rclone/rclone.log \ -vv
# I reduce the poll interval down to 15 seconds as this makes changes appear fast the API quotas per day are huge
--poll-interval 15s \
# This is setting the file permission on the mount to user and group have the same access and other can read
--umask 002 \
# The local disk used for caching
--cache-dir=/cache \
# This is used for caching files to local disk for streaming
--vfs-cache-mode full \
# This limits the cache size to the value below
--vfs-cache-max-size 30G \
# This limits the age in the cache if the size is reached and it removes the oldest files first
--vfs-cache-max-age 336h \
# This sets up the remote control daemon so you can issue rc commands locally
--rc
ExecStop=/bin/fusermount -uz /home/iragoiko/gdrive
Restart=on-success
RestartSec=5
User=iragoiko
Group=iragoiko

[Install]
WantedBy=multi-user.target

Then I run commands:

iragoiko@debian:~/rclone$ sudo systemctl enable rclone.service
iragoiko@debian:~/rclone$ sudo systemctl start rclone.service
iragoiko@debian:~/rclone$ sudo systemctl status rclone.service
● rclone.service - RClone Service
   Loaded: loaded (/etc/systemd/system/rclone.service; enabled; vendor preset: enabled)
   Active: active (running) since Sun 2020-10-25 17:17:36 CET; 7s ago
 Main PID: 3628 (rclone)
    Tasks: 17 (limit: 19660)
   Memory: 23.2M
   CGroup: /system.slice/rclone.service
           └─3628 /usr/bin/rclone mount gdrive: /home/iragoiko/gdrive

All is well mounted

After reboot

rclone.service - RClone Service
   Loaded: loaded (/etc/systemd/system/rclone.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Sun 2020-10-25 17:19:09 CET; 1min 39s ago
  Process: 781 ExecStart=/usr/bin/rclone mount gdrive: /home/iragoiko/gdrive (code=exited, status=1/FAILURE)
 Main PID: 781 (code=exited, status=1/FAILURE)

oct 25 17:19:09 debian systemd[1]: Starting RClone Service...
oct 25 17:19:09 debian rclone[781]: 2020/10/25 17:19:09 Failed to create file system for "gdrive:": couldn't find root directory ID: Get "https://www.googleapis
oct 25 17:19:09 debian systemd[1]: rclone.service: Main process exited, code=exited, status=1/FAILURE
oct 25 17:19:09 debian systemd[1]: rclone.service: Failed with result 'exit-code'.
oct 25 17:19:09 debian systemd[1]: Failed to start RClone Service.

rclone.log (last logs 1 hour ago)

2020/10/25 16:09:22 NOTICE: Serving remote control on http://127.0.0.1:5572/
2020/10/25 16:09:22 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2020/10/25 16:09:43 INFO  : Signal received: terminated
2020/10/25 16:09:43 INFO  : Exiting...
2020/10/25 16:10:40 NOTICE: Serving remote control on http://127.0.0.1:5572/
2020/10/25 16:10:40 Failed to create file system for "gdrive:": couldn't find root directory ID: Get "https://www.googleapis.com/drive/v3/files/root?alt=json&fields=id&prettyPrint=false&supportsAllDrives=true": dial tcp: lookup www.googleapis.com on 192.168.5.1:53: dial udp 192.168.5.1:53: connect: network is unreachable

darthShadow · October 25, 2020, 4:35pm

Your main problem here is that your network is not up and running when rclone starts which renders it unable to resolve DNS.

Try changing the following:

Restart=on-success

to

Restart=on-failure

I am not sure how you came to choose on-success instead? Was that from an official wiki or guide?

Also, try adding this before the ExecStart line:

ExecStartPre=/bin/sleep 5

iragoiko · October 25, 2020, 4:49pm

Yes.

Updated

Restart=on-failure

ExecStartPre=/bin/sleep 5

Now at reboot gdrive is mounted.
The issue is that I can not start docker for plex and emby: I am getting this error at portainer:
error while creating mount source path '/home/iragoiko/gdrive': mkdir /home/iragoiko/gdrive: file exists

Until now I had no issue with these containers.

asdffdsa · October 25, 2020, 4:51pm

hi,
thanks for stopping by.
i have been trying to learn about systemd myself and i was able to get the OP service file working.
but i could not figure out why on boot it was not working.

i have been comparing Animosity022 script and the rclone wiki, which seems overly complex.
is that wiki considered the 'official` guide?

there are a several differences, i was going to start a new post and ask what is better of the two.
perhaps, perchance, maybe thee can comment?

Restart=
Animosity022
Restart=on-failure
rclone wiki
Restart=on-success

Animosity022

Wants=network-online.target
After=network-online.target

rclone wiki

After=network-online.target

KillMode
Animosity022
KillMode=none
rclone wiki
there is no entry, it is missing

ExecStop
Animosity022
ExecStop=/bin/fusermount -uz and then the mount path
rclone wiki
ExecStop=/bin/fusermount -u and then the mount path

thanks in advance,

darthShadow · October 25, 2020, 5:05pm

error while creating mount source path '/home/iragoiko/gdrive': mkdir /home/iragoiko/gdrive: file exists

I don't use portainer myself so not sure where this error is coming from. You will need to explain more about your exact setup and how you start containers.

It should be, considering it's in the official repo wiki.

Coming to the questions,

Restart=on-success

This seems wrong. Not sure why anyone would need to restart the service on success.

After=network-online.target`

This is better in most cases since it doesn't imply starting the service always if the wanted target is started, just that it should be started only after a particular target.

KillMode=none

This is preferred so that if the mount cannot be stopped properly via fusermount, then it doesn't kill the process which may cause issues.

ExecStop=/bin/fusermount -uz

-z just implies not to block on the unmounting of the path.

From the manpage:

-z  lazy unmount.

Just FYI, the systemd docs do a pretty good job of explaining the various options: https://www.freedesktop.org/software/systemd/man/systemd.unit.html

asdffdsa · October 25, 2020, 5:09pm

that is from the official rclone wiki

darthShadow · October 25, 2020, 5:11pm

Yeah, saw that. They probably need to be updated.

@Animosity022 Do you want to take care of that, considering you have done such an excellent job with your guide?

asdffdsa · October 25, 2020, 5:21pm

thanks much, perhaps another question,

when i wrote my service file, i found it a pain to keep systemctl daemon-relaod, doing sudo so on.
so i wrote my service file like so, to run batch files.
that way i can tweak my rclone commands and not have to edit the service file.
is there a downside to that, as it seems a better way to do it?

ExecStart=/bin/bash /home/user/rclone/rclone.mount.start.remote.sh
ExecStop=/bin/bash /home/user/rclone/rclone.mount.stop.remote.sh

Animosity022 · October 25, 2020, 5:27pm

I can take a peek at it. Looks super over complicated and not sure what the creator was going for.

The downside would be you can take full advantage of things that systemd can provide as it's really expecting something to run and you can trigger off of failures by auto restarting things and the journal logging becomes not useful to name a few offhand.

Normally, you aren't changing service files all the time,

asdffdsa · October 25, 2020, 5:30pm

i agree that the current wiki is super complicated.

if you could write a new wiki, optimized for rclone that would be great.
there are so many posts in this forum about systemd confusion.

i tried to compare your guide and that wiki.
as you can see from my posts, i got real confused.

iragoiko · October 25, 2020, 5:31pm

iragoiko@debian:~/gdrive$ docker ps -a
CONTAINER ID        IMAGE                         COMMAND             CREATED             STATUS                        PORTS                                            NAMES
47e3e286ce11        linuxserver/heimdall:latest   "/init"             3 days ago          Up 4 minutes                  443/tcp, 0.0.0.0:9005->80/tcp                    heimdall_heimdall
fb43130aafb4        linuxserver/emby:latest       "/init"             3 days ago          Exited (128) 39 minutes ago   0.0.0.0:8096->8096/tcp, 0.0.0.0:8920->8920/tcp   emby_emby
426e5410187c        linuxserver/plex:latest       "/init"             3 days ago          Exited (128) 43 minutes ago                                                    plex_plex
767f8347b7c4        linuxserver/taisun            "/init"             5 days ago          Up 4 minutes                  0.0.0.0:3001->3000/tcp                           taisun
71985254a249        portainer/portainer-ce        "/portainer"        5 days ago          Up 4 minutes                  0.0.0.0:8000->8000/tcp, 0.0.0.0:9001->9000/tcp   portainer
iragoiko@debian:~/gdrive$ docker start 426e5410187c
Error response from daemon: error while creating mount source path '/home/iragoiko/gdrive': mkdir /home/iragoiko/gdrive: file exists
Error: failed to start containers: 426e5410187c

Could it be some permision issue? before with docker mount command I did not have this issue with containers pointing this mount directory

iragoiko · October 25, 2020, 5:37pm

Do you think so?
With my previous mount command

rclone mount gdrive: /home/iragoiko/gdrive --allow-other --dir-cache-time 96h --vfs-cache-max-age 48h --vfs-read-chunk-size 20M --vfs-read-chunk-size-limit 1G --buffer-size 10M --umask 002 --log-file /home/iragoiko/rclone.log

I never had this issue. Plex and emby where running without issue on the same directory.
Now that service file is running I have this issue.

Need to know where the root issue is and my skills are low for service file and docker

If you are sure service file is not causing this I will need to investigate docker setups.