RClone Config & Upload question

Help and Support
1

Hello guys,

  1. My current media setup explained + question on how to handle upload in the best way.

  2. Optimization of my current rclone mount command.

  1. My current media setup looks like this:
    When I get new content I copy my files from my computer to specific folders on my NAS.

The structure is the following
This is my folder where all my movies are in:
/share/CACHEDEV1_DATA/Multimedia/Video/Movies
This is my folder where all my tv shows are in:
/share/CACHEDEV1_DATA/Multimedia/Video/TVShows

I want to automatically have rclone copy any new contents which will come to those folders to my mounted gcrypt remote with the same folder structure.

How do I do that ?

  1. My current rclone mount is still from when I first set rclone up and I never optimized it since then. I want to use rclone mainly for use with Plex so the loadtimes of my files should be blazingly fast. Also I read a lot about vfs instead of cache but I am not sure on how my command would need to look like.

Because of QNAP NAS I need to specify different locations then the default e.g. cache location because otherwise it tried to put the cache files into the extremly small qnap “root” partion.

rclone mount gcrypt2: /share/CACHEDEV1_DATA/External --allow-other --allow-non-empty --vfs-cache-mode writes --dir-cache-time 24h --cache-chunk-total-size 10G --cache-chunk-path /share/CACHEDEV1_DATA/RCloneCache --cache-chunk-size 10M --cache-tmp-upload-path /share/CACHEDEV1_DATA/RcloneUploads --cache-tmp-wait-time 60m --cache-info-age 28h --cache-db-path /share/CACHEDEV1_DATA/RCloneCache --buffer-size 100M --log-level INFO --log-file /share/CACHEDEV1_DATA/RCloneCache/rclone.log --umask 002 --cache-dir /share/CACHEDEV1_DATA/RCloneCache

This is my current rclone conf

[gdrive2]
type = drive
client_id = CLIENTIDHERE.apps.googleusercontent.com
client_secret = CLIENTSECRETHERE
scope = drive
token = {“access_token”:“ACCESSTOKENHERE”,“token_type”:“Bearer”,“refresh_token”:“1/oq1CO1Xb0pcaJwwab7GsCE9Pe5wz9iT6wZwOrq6N9W1QQjQjg0”,“expiry”:“2018-11-06T20:36:53.547048291+01:00”}

[gcache2]
type = cache
remote = gdrive2:/mediafiles
plex_url = http://IPHERE:32400
plex_username = Plexusernamehere
plex_password = Plexpasswordhere
chunk_size = 5M
info_age = 1d
chunk_total_size = 10G
db_path = /share/CACHEDEV1_DATA/RCloneCache
chunk_path = /share/CACHEDEV1_DATA/RCloneCache
tmp_upload_path = /share/CACHEDEV1_DATA/RcloneUploads
tmp_wait_time = 1h0m0s

[gcrypt2]
type = crypt
remote = gcache2:/crypt
filename_encryption = standard
directory_name_encryption = true
password = gcrypt2passwordhere
password2 = gcrypt2passwordhere

2

Anyone who can help me ?

3

You could use an rclone sync command to keep two areas in sync and just sync directly to the remote and the mount would pick it up in a 1 minute via normal polling.

I don’t use cache though as I use vfs instead.

1 Like
4

Could you show me how my sync command would ne e to look like?

How would I need to modify my configuration to use vfs? I want to change from cache to vfs setup.

5

My flow is a bit different on how I have it setup.

I basically copy locally first to a location and once a night I upload to my google drive.

I’m more for the simplistic approach and less moving parts.

I have majority of my setup configured / documented on my github if you want to take a peek and ask any questions back here.

1 Like
6

Thank you! I will definitely take a look at those later today (I am traveling right now)) and come back here if I have any questions.

7

So as far as I understood I need to remove the google cache part from my config, so it looks like yours (https://github.com/animosity22/homescripts/blob/master/rclone.conf)

When I want to use vfs can I set the location where it puts the buffered files ?
On qnap the problem is that rclone would normally put the buffered stuff into the root directory by default (where also the rclone.conf is stored) and the /root/ directory is on the RAMDISK so space there is very limited and also everything addionally added in the /root/ folder will be gone after reboot (according to qnap support)

Quote from the QNAP Support:
The system is installed on a RAM-disk, so anything added to your /root folder will disappear at reboot.

But I am not sure how my script would need to look like for nightly uploads. I think I need your help with that.

(Hope I could explain myself good enough. Just came home from traveling and I am very exhausted right now. Thats why I am not sure if what I wirte is understandable enough)

8

With VFS, there is nothing stored locally when a file is played. It’s all used in memory and mainly configured by your buffer-size that you have as that is per file.

Do you keep things locally on the QNAP and the goal is to copy/move them to the cloud?

9

Oh okay, thank you for the clarification, so my new rclone mount command could look like this ?

rclone mount gcrypt2: /share/CACHEDEV1_DATA/External --allow-other --allow-non-empty --vfs-cache-mode writes --buffer-size 1G --dir-cache-time 72h --drive-chunk-size 32M --log-level INFO --log-file /share/CACHEDEV1_DATA/rclone/rclone.log --umask 002 --vfs-read-chunk-size 128M --vfs-read-chunk-size-limit off

Right now I want to keep things locally on the QNAP and also have them in the cloud. (Because right now I still have free space on the NAS.)
But I would be great to have an additional folder where I can put TV shows and movies in, which I only want to have in the cloud. (So they get deleted once they are on gdrive).

10

The rclone mount command works perfectly fine now.

Now I only need some advice on how to do the nightly uploads etc. (See quote below)

Thanks a lot for the help you already provided!

11

This would be my command for my first use case:

  /opt/bin/rclone copy /share/CACHEDEV1_DATA/Multimedia/Video/ gcrypt2:Multimedia --checkers 3 --fast-list --log-file /share/CACHEDEV1_DATA/rclone/copy.log -v --tpslimit 3 --transfers 3

Should I add any other flags ?

The above should work right?
My current folder structure is this:

This would be my command for my second use case:

/opt/bin/rclone move /share/CACHEDEV1_DATA/Multimedia/Video/GDrive/ gcrypt2:Multimedia --checkers 3 --fast-list --log-file /share/CACHEDEV1_DATA/rclone/upload.log -v --tpslimit 3 --transfers 3 --delete-empty-src-dirs