Rclone remote service don't start after reboot

[mikelima (mike)]

What is the problem you are having with rclone?

I use rclone with MergerFS for Plex since a year.
I following this tutorial to setting up all of this : Unlimited Plex Storage via Google Drive And Rclone - Muffin's Lab
My rclone config working fine.

But today, after reboot my server, my rclone remote service file don't start.
I run this commands :
sudo systemctl enable gmedia.service
sudo systemctl start gmedia.service

And i have this return error :
Job for gmedia.service failed because a timeout was exceeded.
See "systemctl status gmedia.service" and "journalctl -xeu gmedia.service" for details.

Here is the "journalctl -xeu gmedia.service" return command : https://pastebin.com/ckgp7HKR

Here is the "systemctl status gmedia.service" return command : https://pastebin.com/zB8LBYb9

Nothing was changed in the config. Just reboot the server.

Run the command 'rclone version' and share the full output of the command.

rclone v1.57.0

• os/version: ubuntu 21.10 (64 bit)
• os/kernel: 5.13.0-40-generic (x86_64)
• os/type: linux
• os/arch: amd64
• go/version: go1.17.2
• go/linking: static
• go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Google Drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

sudo systemctl enable gmedia.service
sudo systemctl start gmedia.service

Here is the /etc/systemd/system/gmedia.service file : https://pastebin.com/digSYSuc

The rclone config contents with secrets removed.

[gdrive-media]
type = drive
client_id = XXXXXXXXX.apps.googleusercontent.com
client_secret = XXXXXXXXXXXXX
scope = drive
root_folder_id = XXXXXXXXXXXX
token = {"access_token":"XXXXXXXXXXXX","token_type":"Bearer","refresh_token":"1//0>
team_drive =

[gdrive-media-crypt]
type = crypt
remote = gdrive-media:/media
password = XXXXXXXX
password2 = XXXXXXXX

A log from the command with the -vv flag

Here is the /opt/rclone/logs/rclone-mount.log with "DEBUG" : https://pastebin.com/9GQKRk8c

Thanks for helping me.

[Animosity022]

From your log, it looks like the service is being killed as it's taking too long to start up as it's getting quite a lot of 403 rate limit errors.

You seem to be using a client ID and secret though so that seems a bit odd.

These are making it not quite finish the start:

writeback as --transfers exceeded
2022/05/02 23:21:28 DEBUG : Google drive root 'media': Checking for changes on remote
2022/05/02 23:21:31 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2022/05/02 23:21:31 DEBUG : pacer: Rate limited, increasing sleep to 1.268038371s
2022/05/02 23:21:31 DEBUG : pacer: low level retry 3/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2022/05/02 23:21:31 DEBUG : pacer: Rate limited, increasing sleep to 2.390123447s
2022/05/02 23:21:31 DEBUG : pacer: low level retry 5/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2022/05/02 23:21:31 DEBUG : pacer: Rate limited, increasing sleep to 4.445449266s
2022/05/02 23:21:31 DEBUG : pacer: low level retry 7/10 (error googleapi: Error 403: User rate limit exceeded., userRateLimitExceeded)
2022/05/02 23:21:31 DEBUG : pacer: Rate limited, increasing sleep to 8.284559389s

and since you aren't using KillMode=None in the file, it times out, gets killed and repeats over and over.

Is it some shared account or something or is someone else using the same client ID and secret? If you are using MergerFS, I can't figure out why you uploading stuff either.

[mikelima (mike)]

Thanks for the reply,

I don't know why there is a client id and a secret, I just followed the tutorial here.

I just added killmode=none in the file but it still doesn't seem to start...

It's a Google Workspace account and the client ID and secret are not used elsewhere to my knowledge.

[Animosity022]

You've have to post the new service file, some new logs to see what's going on now.

Are you seeing hits in your Google Admin console for your client ID/secret?

[mikelima (mike)]

Sorry,
Here is the new service file : https://pastebin.com/3J29FNGR
Here is the rclone mount logs : https://pastebin.com/1e3mHQ6z

What do you mean by seeing hits on Admin Console ? Don't really know how to see that in the admin console

[Animosity022]

Where you made the client ID and secret.

Google Cloud Platform

You can see the API hits here around here:

As you want to make sure the Client ID/secret you setup is being used.

[mikelima (mike)]

Thanks,
Here is the screen of the dashboard, it look like to have a lots of errors here ...

[Animosity022]

If you click on the Google Drive API, it should drill down into the next screen. Curious to see what is actually erroring out.

[mikelima (mike)]

here we go

[Animosity022]

You should see something like this:

I want to know what methods are erroring out.

[mikelima (mike)]

[Animosity022]

Is the bottom cut off? Are all the errors in drive.file.create? That generally means you hit an upload quota.

[mikelima (mike)]

Oh sorry, here is the bottom.

So what do i need to retrieve the access on my remote with the service file ? Just wait & see ?

[Animosity022]

The issue in a nutshell.

You have a mount.
Files are waiting to be uploaded.
You hit your daily quota / limit / something and the files can't upload.
They keep trying to upload and failing over and over.

That is going to cause the mount issues as it keeps timing out on the uploads so you'd have to figure out what can be cleared up/moved around so the uploads stop and do that manually later.