Well, I don't use the mount command at all. I use the copy command. So does this still hold? What should I be doing in this case?
Share your service file.
[Unit]
Description=RClone Backup
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
ExecStart=/sh/sync.sh
ExecStop=/bin/kill -s SIGKILL $MAINPID
[Install]
WantedBy=multi-user.target
What's the goal for this service? Does run all the time? What's in the sync.sh?
It is periodic backup triggered by systemd timer - so sync.sh probably have some rclone sync but yes maybe to see the script would help to have better picture.
It's run every two hours by the systemd timer.
Well, the .sh file has the rclone copy command:
rclone copy source: destination: --flags
It is similar to the above command.
what you can do is to add logfile in DEBUG mode to copy command in your script --log-file /path/to/logfile -vv
run your backup service - sudo systemctl start yourBackupSystemd
then do reboot - hopefully logile will show what the issue is
The flag is already included. The log file shows no errors.
And the goal is you want the service to immediately abort on shutdown if it's running?
Yes, that is correct. Even if not taking the literal meaning of 'immediately', the rclone service shouldn't be causing the delay for a normal (usually less than 10s) shutdown.
You'd be better off running it out of cron and scheduling it that way.
The service method wants to nicely stop it and what I would imagine is happening is you are killing the parent PID and the children are probably running still and that's not letting it stop.
Systemd has a long, long timeout and retry which you can tweak, but not really worth the hassle.
If you have it running in cron, it'll just abort if running and reboot.
This is exactly why I suggested systemd as with crontab it was taking ages to do shut down/reboot. However strange for me for rsync copy. With systemd you have much more control. Why it is taking so long to stop it - I am not sure.
And as much as you are right with mount and IO/processes holding it - it does not make sense for me with copy
SIGTERM should stop it immediately. SIGKILL should stop it whatever else is happening.
What children you think are there when running rclone copy? I might be wrong but I do not think rclone spawns any extra processes.
If you have a shell script, it'll spawn new processes for things that run.
Simple test with a kill -9 and a size command.
^C[felix@gemini ~]$ ./blah &
[1] 720712
[felix@gemini ~]$ ps -ef | grep blah
felix 720712 6081 0 17:29 pts/0 00:00:00 /usr/bin/bash ./blah
felix 720754 6081 0 17:29 pts/0 00:00:00 grep --color=auto blah
[felix@gemini ~]$ ps -ef | grep 720712
felix 720712 6081 0 17:29 pts/0 00:00:00 /usr/bin/bash ./blah
felix 720713 720712 17 17:29 pts/0 00:00:01 /usr/bin/rclone size DB:
felix 720776 6081 0 17:29 pts/0 00:00:00 grep --color=auto 720712
[felix@gemini ~]$ kill -9 720712
[felix@gemini ~]$ ps -ef | grep 720713
felix 720713 1 14 17:29 pts/0 00:00:02 /usr/bin/rclone size DB:
felix 720818 6081 0 17:29 pts/0 00:00:00 grep --color=auto 720713
[1]+ Killed ./blah
[felix@gemini ~]$
I had to kill the spawned process.
[felix@gemini ~]$ ps -ef | grep 720713
felix 720713 1 12 17:29 pts/0 00:00:07 /usr/bin/rclone size DB:
felix 721005 6081 0 17:30 pts/0 00:00:00 grep --color=auto 720713
[felix@gemini ~]$ kill 720713
[felix@gemini ~]$ ps -ef | grep 720713
felix 721040 6081 0 17:30 pts/0 00:00:00 grep --color=auto 720713
If the goal was to use systemd, I wouldn't use a script as I'd use the binary if it's a oneshot command.
I generally use crontab for repeat things and systemd for services that run all the time.
@yolobnb could you please share your sh script? You can (or even should) remove all "secrets" from there.
The .sh file has nothing except the rclone copy command. And to check if the service is already running so that the timer doesn't run it again (you know, checking if the PID is running).
Cron is way less controllable than systemd. I have tweaked the systemd timeout, but it isn't a permanent solution as this applies to all services, not just rclone. In my case, it is clear that rclone is failing to stop even with the SIGKILL argument.
Well, I would be happy to use cron, if it is possible to control the rclone service during shutdown. Does Cron have something like the SIGTERM or SIGKILL arguments that can be included in the crontab?
Not sure you saw my example but I showed you it isn't that.
You are killing your script and the child process is still running. As I mentioned above, if all your script is one line, you'd just put that is the service file and your sigkill would work. The system is doing exactly what you told it to do 
As for cron, here's a very simple script that I used forever to upload files when I used mergerfs. It's easier to script a check than mess with systemd imo. You'd want to use whatever works best for you though as my choice might not fly for you.
In the end, it's really a Linux question and nothing related to rclone.
Here is an example without rclone.
You get a dangling process.
[felix@gemini scripts]$ systemctl status test
× test.service - Test Service
Loaded: loaded (/etc/systemd/system/test.service; disabled; preset: disabled)
Drop-In: /usr/lib/systemd/system/service.d
└─10-timeout-abort.conf
Active: failed (Result: signal) since Thu 2023-06-08 19:41:56 EDT; 4s ago
Duration: 40.928s
Process: 754993 ExecStart=/home/felix/scripts/test (code=killed, signal=KILL)
Process: 755187 ExecStop=/bin/kill -s SIGKILL $MAINPID (code=exited, status=0/SUCCESS)
Main PID: 754993 (code=killed, signal=KILL)
Tasks: 1 (limit: 37974)
Memory: 256.0K
CPU: 5ms
CGroup: /system.slice/test.service
└─754994 /usr/bin/sleep 50000
Jun 08 19:41:15 gemini systemd[1]: Started test.service - Test Service.
Jun 08 19:41:56 gemini systemd[1]: Stopping test.service - Test Service...
Jun 08 19:41:56 gemini systemd[1]: test.service: Main process exited, code=killed, status=9/KILL
Jun 08 19:41:56 gemini systemd[1]: test.service: Failed with result 'signal'.
Jun 08 19:41:56 gemini systemd[1]: test.service: Unit process 754994 (sleep) remains running after unit stopped.
Jun 08 19:41:56 gemini systemd[1]: Stopped test.service - Test Service.
[felix@gemini scripts]$ ps -ef | grep 754994
felix 754994 1 0 19:41 ? 00:00:00 /usr/bin/sleep 50000
felix 755252 6081 0 19:42 pts/0 00:00:00 grep --color=auto 754994
[felix@gemini scripts]$ cat /etc/systemd/system/test.service
[Unit]
Description=Test Service
After=network-online.target
[Service]
Type=simple
KillMode=process
Restart=on-abnormal
RestartSec=30
User=felix
Group=felix
ExecStart=/home/felix/scripts/test
ExecStop=/bin/kill -s SIGKILL $MAINPID
[Install]
WantedBy=multi-user.target