Rclone on Proxmox | Plex | Crypt | GDrive

Hi,
I’ve been trying to get rclone to run for a few days, but I can’t get any further.
I have made the following configurations with the instructions given here on a Proxmox LXC.

[gdrive]
type = drive
scope = drive
token = xxx
client_id = xxx
client_secret = xxx

[gcrypt]
type = crypt
remote = gcache:/crypt
filename_encryption = standard
directory_name_encryption = true
password = *** ENCRYPTED ***
password2 = *** ENCRYPTED ***

[gcache]
type = cache
remote = gdrive:/gdrive
plex_url = 192.168.188.101:32400
plex_username = xxx
plex_password = *** ENCRYPTED ***
chunk_size = 5M
info_age = 1d
chunk_total_size = 10G

Proxmox also runs Plex on an LXC (pve 100) and Rclone on a separate LXC (pve 101).
With the following command I only get an error message.

 rclone mount --allow-other --allow-non-empty gcrypt: ~/mnt/gdrive &

Error message:

2019/03/26 22:29:36 Failed to create file system for “gcrypt:”: failed to make remote gcache:"/crypt" to wrap: failed to connect to the Plex API 192.168.188.101:32400: parse 192.168.188.101:32400: invalid URI for request

I have already restarted all containers and the complete server, but unfortunately this didn’t produce any results. No folder is created in the Google Drive and no suitable directory is found in Plex.

My questions are following:

  1. How do I get it to run, do I have to install it on the host (pve), or go another way?
  2. How do I transfer files via rclone?
  3. Can I sort files online if I show the real names during encryption?

Greetings,
clone4ever

That is saying that the plex_url is in the incorrect format. It needs to be a URL something like http://192.168.188.101:32400/ (but I don’t use plex so not 100% sure if that is correct).

That helped, thank you.

But now I have another problem with “FUSE”.

rclone mount --allow-other --allow-non-empty gcrypt: ~/mnt/gdrive &`

Results in

2019/03/27 17:35:20 mount helper error: fusermount: fuse device not found, try 'modprobe fuse' first
2019/03/27 17:35:20 Fatal error: failed to mount FUSE fs: fusermount: exit status 1

So I tried the ‘modprobe fuse’ (On the LXC and Host tried)

modprobe fuse

Results in

modprobe: ERROR: ../libkmod/libkmod.c:586 kmod_search_moddep() could not open moddep file '/lib/modules/4.15.18-9-pve/modules.dep.bin'
modprobe: FATAL: Module fuse not found in directory /lib/modules/4.15.18-9-pve

:confused:

You should check with the proxmox folks as that’s part of their package on how to add it. I’ve never used it and not familiar with it.

Google Foo found this.

I’m using rclone on a Proxmox server and have been doing so for a few years. Because getting FUSE to work correctly in an LXC is a bit cumbersome, I’m handling the rclone mount services on the Proxmox server itself (ie, not in an LXC) and then passing those mounts on to the LXC(s) that need them. This has the added bonus of ensuring that the mounts are up when the server comes up and before it starts any of the containers.

In most, if not all cases, you can do any of your utility functions from within the containers since they won’t need FUSE.

Thank you.
I have now installed it directly on the host (pve) and set it up according to these instructions,
and I skipped the part with Plexdrive.

I also transferred some files for testing purposes.

rclone lsd /mnt/gsuite/media/music
      -1 2019-03-28 01:33:37        -1 GTA Double Pack (2004)
      -1 2019-03-28 01:33:27        -1 GTA IV

In addition, under Plex (LXC), the drive and the created folders are also displayed.

BUT it doesn’t show on the GDrive.
Obviously rclone does not upload the files and only saves the files locally.

Do you know how to fix this?

The mountpoint in the conf file show look something like this:

mp0: /home/kelinger/mnt/Google,mp=/Google

Where the path on the left is where the volume is mounted on the Proxmox host and the path on the right is the location you want that mount to appear within the container. Assuming those are all correct and you’re seeing the files, then what remains is likely an issue of permissions.

From within the LXC, can you “touch” a new file on the remote? IE, in my example above, I’d “touch /Google/ken” and the file will appear on the G-drive as well as in the mount point on the host and container.

I’m using MergerFS overlaying the actual rclone mount so that’s what’s handling all my writes and then a script runs every 15 minutes on the Proxmox host to move those new files to Google so its not 100% apples to apples here. But if you can see the files from within the LXC but can’t write back to them, I’d start looking at how the user permissions are mapped between the host and the container.

Ken

My mountpoint in the conf look like this:

mp7: /mnt/gsuite/media,mp=/gsuite

All my HDD look the same (mp0 - mp6)

I’m not sure what you talking about with the “touch”. When I type

touch gsuite

nothing happend. :confused:


EDIT

I have tried to rework anything for the last 6h, but no better result.

@kelinger do you have an installation guide or something like that for your installation?

OK, with the “touch” command, I was talking about doing this inside the LXC. So, in the LXC itself, as root, you have a /gsuite directory, right?

Can you try:
touch /gsuite/testfile

Does that work or give an error. Do you see the file on the root of /mnt/gsuite/media on the host?

Next, login to your LXC as the non-root user you’re trying to use. Can you try the same command as above?

Does one way work and another way not? If one (or both) work, are you also able to remove the file from within the LXC with
rm /gsuite/testfile

Ken

I have "touch" as you said.
The file can be seen in the host, no errors.
I do not have a non-root user for the LXC.

As I said, I see the file in Plex (and therefore also in the LXC).
The problem is that rclone does not synchronize the files with the GDrive.
So far I've tried this with ~1GB on files, but it doesn't show anything in the cloud.

I'll try it again from the beginning, without the scripts.

EDIT
I have redone the whole setup as discriped here (Without plexdrive and encfs, because rclone does this)

The “touch” is still working
I also managed to copy files to the GDrive (finally).
BUT, I can not see the files or the filesystem of the GDrive :confused:

Also, the files I uploaded are NOT encrypted (I setup the crypt).

rclone copyto -v /mnt/data_2TB_1/folder/ gsuite:media/

EDIT 2
When I type the following line into the host, the Plex LXC is not able to startup

rclone mount --allow-other --allow-non-empty gcrypt: /mnt/gsuite &

After I stop the mount by the following line, Plex LXC will startup again

fusermount -u /mnt/gsuite

I think this is why I can not access the GDrive using Plex … but how solve this problem :thinking:

On the host, what user is running the “rclone mount” command?

The host is running as root

root@pve:~# rclone config
Current remotes:

Name                 Type
====                 ====
gcache               cache
gcrypt               crypt
gsuite               drive

The LXC (100) is also running as root

root@Plex:~#

To get a mount directory for the hard drives and gdrive in the Plex-LXC, I edit the 100.conf as follows

root@pve:/etc/pve/nodes/pve/lxc# nano 100.conf

mp0: /mnt/data1,mp=/data1
mp1: /mnt/data2,mp=/data2
mp2: /mnt/data3,mp=/data3
mp3: /mnt/data4,mp=/data4
mp4: /mnt/data5,mp=/data5
mp5: /mnt/gsuite/media,mp=/gsuite

To mount the all the drives at the start I copy the following text into the host

mount -t ext4 /dev/mapper/data1 /mnt/data1 &&
mount -t ext4 /dev/mapper/data2 /mnt/data2 &&
mount -t ext4 /dev/mapper/data3 /mnt/data3 &&
mount -t ext4 /dev/mapper/data4 /mnt/data4 &&
mount -t ext4 /dev/mapper/data5 /mnt/data5 &&
rclone mount --allow-other --allow-non-empty gcrypt: /mnt/gsuite &

EDIT
For now I have managed to crypt the files (I was to dumb to understand to type in the right commands)
But still can not manage to get the GDrive to be in Plex :frowning:

EDIT2
@kelinger
Is it possible that you copy your settings from Proxmox into this thread (or as PM)?
I don’t know what to do future :confused:

Hi @kelinger, I just want to ask if there is a chance for some more help from you?

First, my apologies. It’s been a busy couple of weeks. I didn’t intend on leaving you hanging!

A moment of truth: I stopped using LXC for my Plex configuration about a year ago and went with Docker. However, I did have this mapped and working from LXC so I’ve recreated this configuration for the purposes of this response. In doing so, I discovered that the important step I didn’t mention earlier is that the container be created in “privileged” access, not unprivileged. There is likely a method to get unprivileged working, but it was either specific to UIDs or added a lot of steps or both.

For the sake of openness, here is my configuration:

Proxmox: latest version (currently 5.3-12)
LXC: Using the Debian 9.7 template, privileged

From the LXC template in /etc/pve/lxc (101.conf in my case)
mp0: /home/kelinger/Omni/mnt/Google,mp=/Google

Basically, on the host, I have my Google mounted with rclone mount commands to /home/kelinger/Omni/mnt/GoogleCrypt and then remount it with MergerFS and some local directories to /home/kelinger/Omni/mnt/Google. Thus, I’m passing that read-write directory from the host to /Google inside the LXC.

On the host, ls -l /home/kelinger/Omni/mnt/Google:
drwxr-xr-x 6 kelinger kelinger 6 Apr 8 15:28 Downloads/
drwxrwxr-x 2 kelinger kelinger 2 Apr 10 11:15 Media/

Once in the LXC (from the host, pct enter 101)
ls -l /Google:
drwxr-xr-x 6 1000 1000 6 Apr 8 22:28 Downloads
drwxrwxr-x 2 1000 1000 2 Apr 10 18:15 Media

Now, inside the LXC (as root as I haven’t done anything beyond create the LXC and connect to it at this point), I type touch /Google/testfile

And my directory now looks like:
drwxr-xr-x 6 1000 1000 6 Apr 8 22:28 Downloads
drwxrwxr-x 2 1000 1000 2 Apr 10 18:15 Media
-rw-r--r-- 1 root root 0 Apr 10 19:37 testfile

Whereas on the host, it looks like:
drwxr-xr-x 6 kelinger kelinger 6 Apr 8 15:28 Downloads/
drwxrwxr-x 2 kelinger kelinger 2 Apr 10 11:15 Media/
-rw-r--r-- 1 root root 0 Apr 10 12:37 testfile

So basically, in privileged mode, it’s mapping the LXC’s UIDs and GIDs to those on the host. I don’t have a user/group 1000 yet, but if I create one (adduser lxcuser from within the LXC) my directory listing magically turns into:
drwxr-xr-x 6 lxcuser lxcuser 6 Apr 8 22:28 Downloads
drwxrwxr-x 2 lxcuser lxcuser 2 Apr 10 18:15 Media
-rw-r--r-- 1 root root 0 Apr 10 19:37 testfile

(this is assuming that the user created has UID and GID 1000; you can create the group ahead of time and use parameters to force adduser to use 1000/1000 if those are available, which they should be on a fresh LXC).

A quick test of rm testfile shows that I can also delete files from the mount within the LXC.

Are you able to replicate the process above on your system successfully?

Thank you for the response, I will try it the next days.
I will give feedback after testing :slight_smile:

After a few testings I managed to get it working in the right way.

But for now I have one more problem.
If I run rclone to mount the GSuite for Plex, the upload of other files stuck after a few seconds.
As mentioned in an other thread, I have created a new 'gcrypt' config in rclone just for the upload.

Is there something I have to look change that the upload do not interrupt after a few minutes?
Or can I see what is the problem anywhere?

Is the download or upload using all the bandwidth? You might want to use --bwlimit in that case.

It look like this is a problem of proxmox again.
After the logout the upload is stop working :confused: need to find another solution.

Thanks so far :slight_smile: