Rclone mount to slow scan

el_pedriyo · April 2, 2019, 7:37pm

Hello,

I just wanted to point out that some third parties services I use with the mounted google drive unit are lasting to much to scan my files, is there any reason for that and is there any solution?

Kind regards

calisro · April 2, 2019, 7:47pm

What command area you using to mount? And a ‘-vv’ debug log would help.

el_pedriyo · April 2, 2019, 8:29pm

Hello,

I have using the following to mount it:

  GNU nano 2.7.4                                                      Fichero: /etc/systemd/system/rclone.service

# /etc/systemd/system/rclone.service
[Unit]
Description=Google Drive (rclone)
AssertPathIsDirectory=/home/pedro/series_gd
After=plexdrive.service

[Service]
Type=simple
ExecStart=/usr/bin/rclone mount \
        --config=/root/.config/rclone/rclone.conf \
        --allow-other \
        --cache-tmp-upload-path=/tmp/rclone/upload \
        --cache-chunk-path=/tmp/rclone/chunks \
        --cache-workers=8 \
        --cache-writes \
        --cache-dir=/tmp/rclone/vfs \
        --cache-db-path=/tmp/rclone/db \
        --no-modtime \
        --drive-use-trash \
        --stats=0 \
        --checkers=16 \
        --bwlimit=4000M \
        --dir-cache-time=60m \
        --cache-info-age=60m encrypt:/series /home/pedro/series_gd
ExecStop=/bin/fusermount -u /home/pedro/series_gd
Restart=always
RestartSec=10

[Install]
WantedBy=default.target

Kind regards

el_pedriyo · April 2, 2019, 8:30pm

I will try with the debug to see if something pops up.

Kind regards

Animosity022 · April 2, 2019, 8:36pm

rclone version shows what?

Are you using your own API key / client iD?

https://rclone.org/drive/#making-your-own-client-id

checkers does nothing on a mount so it can be removed.

You have your bandwidth limited which would slow things down.

Your dir-cache time is small at 60 minutes and your cache-info-age should be greater than the dir-cache-time.

el_pedriyo · April 2, 2019, 9:25pm

Hello,

root@services:/usr/local/sbin# rclone version
rclone v1.46

os/arch: linux/amd64
go version: go1.11.5

I have run the rclone mount with this command:

rclone mount -vv encrypt:series /home/pedro/series_gd/

And here is the output:
https://pastebin.com/cxZqYJSd

As you can see it list 1 file per 1 second more or less

el_pedriyo · April 2, 2019, 9:26pm

Hello,

Also I imagine something similar happens with plex when scanning the google drive as it also last a lot of time.

Kind regards

calisro · April 2, 2019, 9:35pm

Personally I’d increase you dir-cache-time and cache-info_time. Changes to the mounts will still be done with-in the notify interval from the Google change log API. That’ll make listings much faster. Just like he said. And your own API…

Also we’ve found that empty directories on your remote can cause the cache to make API calls so you should do a rmdirs on the remote to delete empty ones of there are any.

Lastly do you really need cache rather than just vfs?

el_pedriyo · April 2, 2019, 9:58pm

Helo,

Sorry for the confusion, I think I really do not need cache, I mean, it is just for displaying video through my system, not other type of small files I access a lot. For the API part I am already working with mine.
And about the empty directories, as far as I know I do not have those directories on my google drive, I think the third party service might be asking for them.
This is my current folder structure:

root@services:/usr/local/sbin# rclone ls encrypt:series
1689557325 Hakan, el protector/Hakan, el protector - 1x01 - Episodio 1.mkv
1236792320 Hakan, el protector/Hakan, el protector - 1x02 - Episodio 2.mkv
1318672546 Hakan, el protector/Hakan, el protector - 1x03 - Episodio 3.mkv
1673087329 Hakan, el protector/Hakan, el protector - 1x04 - Episodio 4.mkv
1697831258 Hakan, el protector/Hakan, el protector - 1x05 - Episodio 5.mkv
1417452875 Hakan, el protector/Hakan, el protector - 1x06 - Episodio 6.mkv
1544919686 Hakan, el protector/Hakan, el protector - 1x07 - Episodio 7.mkv
1548454313 Hakan, el protector/Hakan, el protector - 1x08 - Episodio 8.mkv
1298544927 Hakan, el protector/Hakan, el protector - 1x09 - Episodio 9.mkv
1170908782 Hakan, el protector/Hakan, el protector - 1x10 - Episodio 10.mkv
2530209460 Jack Ryan/Jack Ryan, de Tom Clancy - 1x01 - Piloto.mkv
1759031020 Jack Ryan/Jack Ryan, de Tom Clancy - 1x02 - Conexión francesa.mkv
1606783904 Jack Ryan/Jack Ryan, de Tom Clancy - 1x03 - 22 Negro.mkv
1340240953 Jack Ryan/Jack Ryan, de Tom Clancy - 1x04 - El Lobo.mkv
1535545016 Jack Ryan/Jack Ryan, de Tom Clancy - 1x05 - El fin del honor.mkv
2101424614 Jack Ryan/Jack Ryan, de Tom Clancy - 1x06 - Fuentes y métodos.mkv
1787266055 Jack Ryan/Jack Ryan, de Tom Clancy - 1x07 - El muchacho.mkv
1597745700 Jack Ryan/Jack Ryan, de Tom Clancy - 1x08 - Si Dios quiere.mkv
1929378915 Dirty John/Dirty John - 1x01 - Episodio 1.mkv
1774931763 Dirty John/Dirty John - 1x02 - Episodio 2.mkv
1805772127 Dirty John/Dirty John - 1x03 - Episodio 3.mkv
1617210618 Dirty John/Dirty John - 1x04 - Episodio 4.mkv
1805209794 Dirty John/Dirty John - 1x05 - Episodio 5.mkv
1549905612 Dirty John/Dirty John - 1x06 - Episodio 6.mkv
1581562314 Dirty John/Dirty John - 1x07 - Episodio 7.mkv
1614938240 Dirty John/Dirty John - 1x08 - Episodio 8.mkv
2228958636 Bodyguard/Bodyguard - 1x01 - Episodio 1.mkv
2249565972 Bodyguard/Bodyguard - 1x02 - Episodio 2.mkv
2249766647 Bodyguard/Bodyguard - 1x03 - Episodio 3.mkv
2265986473 Bodyguard/Bodyguard - 1x04 - Episodio 4.mkv
2293642119 Bodyguard/Bodyguard - 1x05 - Episodio 5.mkv
3007854446 Bodyguard/Bodyguard - 1x06 - Episodio 6.mkv
2813628263 Estoy Vivo/Estoy Vivo - 1x01 - La oportunidad.mkv
2885701146 Estoy Vivo/Estoy Vivo - 1x02 - No es humano.mkv
2915346908 Estoy Vivo/Estoy Vivo - 1x03 - De mito a hombre.mkv
3123363324 Estoy Vivo/Estoy Vivo - 1x04 - Llorar es de fuertes.mkv
3003402749 Estoy Vivo/Estoy Vivo - 1x05 - Más cerca que nunca.mkv
2971611877 Estoy Vivo/Estoy Vivo - 1x06 - La vida sigue.mkv
3306150068 Estoy Vivo/Estoy Vivo - 1x07 - Te echo de menos.mkv
3014897988 Estoy Vivo/Estoy Vivo - 1x08 - Un cielo sin colores.mkv
3042390556 Estoy Vivo/Estoy Vivo - 1x09 - No me olvides.mkv
3470077795 Estoy Vivo/Estoy Vivo - 1x10 - Lo que debemos hacer.mkv
2916993433 Estoy Vivo/Estoy Vivo - 1x11 - Por última vez.mkv
3037556143 Estoy Vivo/Estoy Vivo - 1x12 - Tengo algo que decirte.mkv
3404591327 Estoy Vivo/Estoy Vivo - 1x13 - Cuestión de sangre.mkv
3462858324 Estoy Vivo/Estoy Vivo - 2x01 - El Rey de la Baraja.mkv
2876466413 Estoy Vivo/Estoy Vivo - 2x02 - Secretos compartidos.mkv
3045717749 Estoy Vivo/Estoy Vivo - 2x03 - El día después.mkv
3031689829 Estoy Vivo/Estoy Vivo - 2x04 - La llave.mkv
2923039532 Estoy Vivo/Estoy Vivo - 2x05 - Ángela.mkv
2917057167 Estoy Vivo/Estoy Vivo - 2x06 - Un hombre de palabra.mkv
2865117265 Estoy Vivo/Estoy Vivo - 2x07 - Entre dos mundos.mkv
2995166342 Estoy Vivo/Estoy Vivo - 2x08 - La caja.mkv
2861758131 Estoy Vivo/Estoy Vivo - 2x09 - Mentiras y latidos.mkv
2902170449 Estoy Vivo/Estoy Vivo - 2x10 - Relaciones personales.mkv
2843144220 Estoy Vivo/Estoy Vivo - 2x11 - El punto de partida.mkv
2774037101 Estoy Vivo/Estoy Vivo - 2x12 - Un paso adelante.mkv
3218637284 Estoy Vivo/Estoy Vivo - 2x13 - Dormíamos despiertos.mkv

Kind regards

el_pedriyo · April 2, 2019, 10:05pm

Hello,

Here is my debug when using plex with the mount unit:

https://pastebin.com/4mQzLMbk

As you can see here there is no errors, but it seems it also takes a while to scan the files.

Kind regards

Animosity022 · April 3, 2019, 12:13am

I’m not sure what you mean by slow as that looks very normal. Analyzing in plex requires opening a file and reading part of it.

Those logs look solid with no errors. Once things are analyzed in Plex, it just checks the file size/timestamps and doesn’t do anything else.

I think I scanned a 40TB library or so in about 2-3 days so seems good.

What are you expecting to happen?

system · July 2, 2019, 12:13am

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.