Rclone mounted but not working

gelsas · September 5, 2018, 2:33am

I mounted my gcache and set some options but it seems it is not uploading from the cache tmp upload path.

Below are my current cofigs and my mount command.

[gdrive2]
type = drive
client_id = CLIENTIDHERE
client_secret = CLIENTSECRETHERE
scope = drive
token = {“access_token”:“TOKENHERE”,“token_type”:“Bearer”,“refresh_token”:“STUFFHERE”,“expiry”:“2018-09-05T04:44:42.822412969+02:00”}

[gcache2]
type = cache
remote = gdrive2:/mediafiles
plex_url = http://IP:32400
plex_username = USERPLEX
plex_password = PASSPLEX
chunk_size = 5M
info_age = 1d
chunk_total_size = 10G

[gcrypt2]
type = crypt
remote = gcache2:/crypt
filename_encryption = standard
directory_name_encryption = true
password = PASS1
password2 = PASS2

rclone mount gcrypt2: /share/CACHEDEV1_DATA/External
–allow-other
–allow-non-empty
–dir-cache-time 24h
–cache-chunk-total-size 5G
–cache-chunk-size 10M
–cache-tmp-upload-path /share/CACHEDEV1_DATA/RcloneUploads
–cache-tmp-wait-time 60m
–cache-info-age 28h
–cache-workers 5
–buffer-size 100M
–log-level INFO
–log-file /share/CACHEDEV1_DATA/Public/rclone-test.log
–umask 002

Shouldn’t now files which are put in RcloneUploads be uploaded after 60m ?

And just to be sure in Plex I need to point to /share/CACHEDEV1_DATA/External correct?

regards

gelsas · September 5, 2018, 12:10pm

This is the rclone-test.log content:

2018/09/05 02:48:17 INFO : gcache2: Cache DB path: /root/.cache/rclone/cache-backend/gcache2.db
2018/09/05 02:48:17 INFO : gcache2: Cache chunk path: /root/.cache/rclone/cache-backend/gcache2
2018/09/05 02:48:17 INFO : gcache2: Chunk Memory: true
2018/09/05 02:48:17 INFO : gcache2: Chunk Size: 10M
2018/09/05 02:48:17 INFO : gcache2: Chunk Total Size: 5G
2018/09/05 02:48:17 INFO : gcache2: Chunk Clean Interval: 1m0s
2018/09/05 02:48:17 INFO : gcache2: Workers: 5
2018/09/05 02:48:17 INFO : gcache2: File Age: 1.1666666666666667d
2018/09/05 02:48:17 INFO : gcache2: Cache Writes: enabled
2018/09/05 02:48:17 INFO : gcache2: Upload Temp Rest Time: 1h0m0s
2018/09/05 02:48:17 INFO : gcache2: Upload Temp FS: /share/CACHEDEV1_DATA/RcloneUploads
2018/09/05 02:48:17 INFO : gcache2: Cache DB path: /root/.cache/rclone/cache-backend/gcache2.db
2018/09/05 02:48:17 INFO : gcache2: Cache chunk path: /root/.cache/rclone/cache-backend/gcache2
2018/09/05 02:48:17 INFO : gcache2: Chunk Memory: true
2018/09/05 02:48:17 INFO : gcache2: Chunk Size: 10M
2018/09/05 02:48:17 INFO : gcache2: Chunk Total Size: 5G
2018/09/05 02:48:17 INFO : gcache2: Chunk Clean Interval: 1m0s
2018/09/05 02:48:17 INFO : gcache2: Workers: 5
2018/09/05 02:48:17 INFO : gcache2: File Age: 1.1666666666666667d
2018/09/05 02:48:17 INFO : gcache2: Cache Writes: enabled
2018/09/05 02:48:17 INFO : gcache2: Upload Temp Rest Time: 1h0m0s
2018/09/05 02:48:17 INFO : gcache2: Upload Temp FS: /share/CACHEDEV1_DATA/RcloneUploads
2018/09/05 02:48:58 INFO : Cache remote gcache2:crypt: Clearing cache from signal
2018/09/05 02:48:58 INFO : Cache remote gcache2:crypt: Clearing cache from signal

gelsas · September 5, 2018, 12:11pm

And as far as I can see the process is running right ? What did I do wrong ?

[~] # ps -ef | grep rclone
6411 admin 10908 S rclone mount gcrypt2: /share/CACHEDEV1_DATA/External --allow-other --allow-non-empty --dir-cache-time 24h --cache-chunk-total-size 5G --cache-chunk-size 10M --cache-tmp-upload-path /share/CACHEDEV1_DATA/RcloneUploads --cache-tmp-wait-time 60m --cache-info-age 28h --cache-workers 5 --buffer-size 100M --log-level INFO --log-file /share/CACHEDEV1_DATA/Public/rclone-test.log --umask 002
12764 admin 1000 S grep rclone

Animosity022 · September 5, 2018, 2:33pm

How are you copying files?

Can you share what command you used?

gelsas · September 5, 2018, 2:52pm

Well I have opended in windows the “External” Folder (I mean this one: /share/CACHEDEV1_DATA/External) and then drag and dropped a file into it.

But windows promts this error after a while: Error 0x8007003B

This is the log:
2018/09/05 16:50:21 ERROR : ntb4d0g9o9rguprf80ftg3i258/bnuea1v8diu47t55suhr2fv2g0: error refreshing object in : in parent fs Local file system at /share/CACHEDEV1_DATA/RcloneUploads: object not found
2018/09/05 16:50:21 ERROR : ntb4d0g9o9rguprf80ftg3i258/bnuea1v8diu47t55suhr2fv2g0: error refreshing object in : in parent fs Local file system at /share/CACHEDEV1_DATA/RcloneUploads: object not found
2018/09/05 16:50:21 ERROR : ntb4d0g9o9rguprf80ftg3i258/bnuea1v8diu47t55suhr2fv2g0: error refreshing object in : in parent fs Local file system at /share/CACHEDEV1_DATA/RcloneUploads: object not found
2018/09/05 16:50:21 ERROR : ntb4d0g9o9rguprf80ftg3i258/bnuea1v8diu47t55suhr2fv2g0: error refreshing object in : in parent fs Local file system at /share/CACHEDEV1_DATA/RcloneUploads: object not found
2018/09/05 16:50:21 ERROR : ntb4d0g9o9rguprf80ftg3i258/bnuea1v8diu47t55suhr2fv2g0: error refreshing object in : in parent fs Local file system at /share/CACHEDEV1_DATA/RcloneUploads: object not found
2018/09/05 16:50:21 ERROR : ntb4d0g9o9rguprf80ftg3i258/bnuea1v8diu47t55suhr2fv2g0: error refreshing object in : in parent fs Local file system at /share/CACHEDEV1_DATA/RcloneUploads: object not found
2018/09/05 16:50:21 ERROR : ntb4d0g9o9rguprf80ftg3i258/bnuea1v8diu47t55suhr2fv2g0: error refreshing object in : in parent fs Local file system at /share/CACHEDEV1_DATA/RcloneUploads: object not found
2018/09/05 16:50:21 ERROR : ntb4d0g9o9rguprf80ftg3i258/bnuea1v8diu47t55suhr2fv2g0: error refreshing object in : in parent fs Local file system at /share/CACHEDEV1_DATA/RcloneUploads: object not found
2018/09/05 16:50:21 ERROR : ntb4d0g9o9rguprf80ftg3i258/bnuea1v8diu47t55suhr2fv2g0: error refreshing object in : in parent fs Local file system at /share/CACHEDEV1_DATA/RcloneUploads: object not found
2018/09/05 16:50:21 ERROR : ntb4d0g9o9rguprf80ftg3i258/bnuea1v8diu47t55suhr2fv2g0: error refreshing object in : in parent fs Local file system at /share/CACHEDEV1_DATA/RcloneUploads: object not found
2018/09/05 16:50:21 ERROR : ntb4d0g9o9rguprf80ftg3i258/bnuea1v8diu47t55suhr2fv2g0: error refreshing object in : in parent fs Local file system at /share/CACHEDEV1_DATA/RcloneUploads: object not found
2018/09/05 16:50:21 ERROR : ntb4d0g9o9rguprf80ftg3i258/bnuea1v8diu47t55suhr2fv2g0: error refreshing object in : in parent fs Local file system at /share/CACHEDEV1_DATA/RcloneUploads: object not found
2018/09/05 16:50:27 INFO : a0vgf0asj1ptejcoqq1qm0404g: mkdir: cache expired
2018/09/05 16:50:32 INFO : a0vgf0asj1ptejcoqq1qm0404g/jtv0nlv5lco684makhehj75foeaeqsfcbttk3255koioecb9ibag: mkdir: cache expired
2018/09/05 16:51:18 INFO : Shows/BNN/S1: received cache expiry notification
2018/09/05 16:52:27 INFO : Shows/BNN: received cache expiry notification
2018/09/05 16:52:39 INFO : a0vgf0asj1ptejcoqq1qm0404g/jtv0nlv5lco684makhehj75foeaeqsfcbttk3255koioecb9ibag/di5nv84816itlhdt8p4ej7049s/6g6hgtadeut1jho8p0khtrqm9jo4e38llr4umc2p912i1nb59qvg: put: uploaded in temp fs
2018/09/05 16:52:40 INFO : a0vgf0asj1ptejcoqq1qm0404g/jtv0nlv5lco684makhehj75foeaeqsfcbttk3255koioecb9ibag/di5nv84816itlhdt8p4ej7049s/6g6hgtadeut1jho8p0khtrqm9jo4e38llr4umc2p912i1nb59qvg: put: queued for upload
2018/09/05 16:52:41 INFO : a0vgf0asj1ptejcoqq1qm0404g/jtv0nlv5lco684makhehj75foeaeqsfcbttk3255koioecb9ibag/di5nv84816itlhdt8p4ej7049s: put: cache expired
2018/09/05 16:52:41 INFO : Shows/BNN/S1/mediafile.mkv: Copied (new)

This is my mount command:
rclone mount gcrypt2: /share/CACHEDEV1_DATA/External
–allow-other
–allow-non-empty
–vfs-cache-mode writes
–dir-cache-time 24h
–cache-chunk-total-size 10G
–cache-chunk-path /share/CACHEDEV1_DATA/RCloneCache
–cache-chunk-size 10M
–cache-tmp-upload-path /share/CACHEDEV1_DATA/RcloneUploads
–cache-tmp-wait-time 60m
–cache-info-age 28h
–cache-db-path /share/CACHEDEV1_DATA/RCloneCache
–buffer-size 100M
–log-level INFO
–log-file /share/CACHEDEV1_DATA/Public/rclone.log
–umask 002
–cache-dir /share/CACHEDEV1_DATA/RCloneCache

Animosity022 · September 5, 2018, 2:55pm

So that means it looked in the cache-tmp-upload area and the file wasn’t there anymore.

I’m not sure why unless you cleaned it up, changed the path or something along those lines.

gelsas · September 5, 2018, 3:12pm

Can I Purge it somehow so I will not get those errors?
I just changed the folder names of the folderw I created inside of the External Folder because I had a typo

But I think that is not the reason for the error from Windows right ?

Animosity022 · September 5, 2018, 3:17pm

You’d have to either remove the cache.db or start the mount with a cache-purge option to clean it up and remove all the files from the cache-tmp-upload

gelsas · September 5, 2018, 3:36pm

Okay after deleting the cache db now the log looks like this:

2018/09/05 17:29:13 INFO : gcache2: Cache DB path: /share/CACHEDEV1_DATA/RCloneCache/gcache2.db
2018/09/05 17:29:13 INFO : gcache2: Cache chunk path: /share/CACHEDEV1_DATA/RCloneCache/gcache2
2018/09/05 17:29:13 INFO : gcache2: Chunk Memory: true
2018/09/05 17:29:13 INFO : gcache2: Chunk Size: 10M
2018/09/05 17:29:13 INFO : gcache2: Chunk Total Size: 10G
2018/09/05 17:29:13 INFO : gcache2: Chunk Clean Interval: 1m0s
2018/09/05 17:29:13 INFO : gcache2: Workers: 4
2018/09/05 17:29:13 INFO : gcache2: File Age: 1.1666666666666667d
2018/09/05 17:29:13 INFO : gcache2: Cache Writes: enabled
2018/09/05 17:29:13 INFO : gcache2: Upload Temp Rest Time: 1h0m0s
2018/09/05 17:29:13 INFO : gcache2: Upload Temp FS: /share/CACHEDEV1_DATA/RcloneUploads
2018/09/05 17:29:14 INFO : gcache2: Cache DB path: /share/CACHEDEV1_DATA/RCloneCache/gcache2.db
2018/09/05 17:29:14 INFO : gcache2: Cache chunk path: /share/CACHEDEV1_DATA/RCloneCache/gcache2
2018/09/05 17:29:14 INFO : gcache2: Chunk Memory: true
2018/09/05 17:29:14 INFO : gcache2: Chunk Size: 10M
2018/09/05 17:29:14 INFO : gcache2: Chunk Total Size: 10G
2018/09/05 17:29:14 INFO : gcache2: Chunk Clean Interval: 1m0s
2018/09/05 17:29:14 INFO : gcache2: Workers: 4
2018/09/05 17:29:14 INFO : gcache2: File Age: 1.1666666666666667d
2018/09/05 17:29:14 INFO : gcache2: Cache Writes: enabled
2018/09/05 17:29:14 INFO : gcache2: Upload Temp Rest Time: 1h0m0s
2018/09/05 17:29:14 INFO : gcache2: Upload Temp FS: /share/CACHEDEV1_DATA/RcloneUploads
2018/09/05 17:30:54 INFO : : mkdir: cache expired
2018/09/05 17:31:00 INFO : o0hnpvmlq68lkiudng5s3ir2vg: mkdir: cache expired
2018/09/05 17:31:03 INFO : o0hnpvmlq68lkiudng5s3ir2vg/jtv0nlv5lco684makhehj75foeaeqsfcbttk3255koioecb9ibag: mkdir: cache expired
2018/09/05 17:31:16 INFO : o0hnpvmlq68lkiudng5s3ir2vg: received cache expiry notification
2018/09/05 17:31:17 INFO : TV-Shows: received cache expiry notification
2018/09/05 17:31:17 INFO : TV-Shows/BNN: received cache expiry notification
2018/09/05 17:31:17 INFO : TV-Shows/BNN/S1: received cache expiry notification
2018/09/05 17:33:20 INFO : : received cache expiry notification
2018/09/05 17:33:20 INFO : ntb4d0g9o9rguprf80ftg3i258: received cache expiry notification
2018/09/05 17:33:21 INFO : : received cache expiry notification
2018/09/05 17:33:21 INFO : @Recycle: received cache expiry notification
2018/09/05 17:33:30 INFO : o0hnpvmlq68lkiudng5s3ir2vg/jtv0nlv5lco684makhehj75foeaeqsfcbttk3255koioecb9ibag/di5nv84816itlhdt8p4ej7049s/6g6hgtadeut1jho8p0khtrqm9jo4e38llr4umc2p912i1nb59qvg: put: uploaded in temp fs
2018/09/05 17:33:30 INFO : o0hnpvmlq68lkiudng5s3ir2vg/jtv0nlv5lco684makhehj75foeaeqsfcbttk3255koioecb9ibag/di5nv84816itlhdt8p4ej7049s/6g6hgtadeut1jho8p0khtrqm9jo4e38llr4umc2p912i1nb59qvg: put: queued for upload
2018/09/05 17:33:30 INFO : o0hnpvmlq68lkiudng5s3ir2vg/jtv0nlv5lco684makhehj75foeaeqsfcbttk3255koioecb9ibag/di5nv84816itlhdt8p4ej7049s: put: cache expired
2018/09/05 17:33:30 INFO : TV-Shows/BNN/S1/mediafile.mkv: Copied (new)

Just from looking at the log it seems that it worked but that is not the case because Windows promted me with this error: Error 0x8007003B An unexpected network error occurred

(But the NAS is accesible the whole time via the network)

What can I do now ?

What I basically want to do is to use that to /share/CACHEDEV1_DATA/External mounted gcrypt2 remote for Plex but I think I am doing something completly wrong because I am running into constant errors

Animosity022 · September 5, 2018, 3:58pm

Sorry as I’m not a windows guy so I’m not sure what that error would mean.

You can add the option to the mount:

--cache-db-purge Purge the cache DB before

but if you stopped the mount, deleted the cache.db, it achieves the same goal as adding that line in.

For me, it looks like it is working from the logs as it copied over as you can see with the ‘new’ item and it’s queued for upload so all looks ok.

I’d wait to see if it uploads based on your time as that should generate another log message.

gelsas · September 5, 2018, 4:08pm

So normally this should already show up in my plex an be playable ?

Just once again a question if my setup is correct.

On my NAS:
I have applicatons running which are getting mediafiles and they get saved to /share/CACHEDEV1_DATA/Mediafiles.

I want those mediafiles to be uploaded encrypted to my google drive but it should work with Plex.

Thats why I created the gdrive2 remote, gcache2 and my gcrypt2.

Then I mounted my gcrypt2 to /share/CACHEDEV1_DATA/External and from what I understood I now need to put the files from /share/CACHEDEV1_DATA/Mediafiles to share/CACHEDEV1_DATA/External so they will get uploaded to google drive (and deleted from my NAS once it is done).

Then I pointed my Plex to share/CACHEDEV1_DATA/External/

And the files should be retrieved from google drive when I try to play them in Plex then, correct?

Is this process correct?

gelsas · September 6, 2018, 12:04am

I think it was just an windows issue with copying. How can I verify that the files in my mounted folder are really not taking anyspace from my NAS?