Rclone remote cache

I would like to switch to rclone cache possibly vfs. I currently have plexdrive non encrypted on gdrive all 100% working but people recommend vfs or cache rclone.

If I go about doing this just the cache be encrypted.

If so is there a simple way to download and re upload to encrypted. Anyway to do remote to remote if so how would I go about doing this ??

Once this is done to setup the gdrive do I point it somewhere in config ? And what would be good commands for mount ?

On side note what is a good command to upload to gdrive as I can not seem to upload to plexdrive currently so I am not sure exactly how to upload to gdrive

Thanks

What is your rclone version (output from rclone version)

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Which cloud storage system are you using? (eg Google Drive)

The command you were trying to run (eg rclone copy /tmp remote:tmp)

A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)

To answer at least some of this:

To encrypt already uploaded data there is no way to do this on-server. It has to be downloaded and re-uploaded. You can either do this yourself locally, or use a VPS for the job if your bandwidth is limited. Google cloud's GCE (basically VMs hosted in googles network) is pretty ideal for this, especially as you get a free bunch of credits to start off and all bandwidth inside googles network is free (which means the traffic to/from your drive is not only blazing fast but also costs nothing extra).

Actually doing the procedure would simlpy involve setting up 2 remotes. one unencrypted and one encrypted - then moving (or copying if you want) from the unencrypted one into the encrypted (preferably to some kind of separate folder structure for the sake of keeping track of your progress). Doing this on the same drive isn't a problem.

VPS or not, you will still have to deal with a 750GB/day quota.

About cache.
to encrypt cache, simply set it up like
Crypt --> Cache --> Gdrive (this is in any case the recommended ordering)
The way you stack like this is simply to set the location of the Crypt to point to the remote-name of the cache, and the same for cache (pointing it to the remote name of the Gdrive). Traffic then goes up and down through those layers.

Uploading on Gdrive: youd have to give specifics about what your problem is because a totally default setup allows uploading just fine and you shouldn't have to do anything special.

Thank you for this. I will setup cache on the drive, so I have to have encryption on it or just use it with just rclone cache. Unencrypted.

The upload command I use is:
/home/username/media/Movies gdrive:movies

Or would I need the gdrive to be gdrive: /username/plexdrive/movies ?

That is where I am now confused also what commands would I use for fast upload thought before it was —L —P

Thanks so far

With gdrive cloud server to do transfer I just pick the gce then rclone move remote: remote:

I don't really understand what you are trying to show here. This is not a command, but rather two locations. You could use them in a command if that's what you mean... like:
rclone move /home/username/media/Movies gdrive:movies
That should work perfectly fine.

Don't use space between gdrive: and the rest of the path. Aside from that the syntax should be ok. I think gdrive:movies and grdive:/movies/ are both valid and work the same.

You don't need to put them anywhere in particular... that's up to you. The part behind gdrive: is just a folder location on the google drive, just the same as on your local harddrive. You can move files around on the gdrive later as needed without having to reupload, so if you need them in a spesific place for Plex then do that as you will.

To upload you just use move, copy or sync commands to the gdrive, like the example above
rclone move /home/username/media/Movies gdrive:movies

There isn't a lot you need to think about in terms of optimized performance, but here are some options I would recommend using:
--fast-list (this will decrease the time it takes to list the drive before uploading starts)
--progress (this will give you a real-time display of the transfers, how much is left ect.)
--drive-chunk-size 128M (this will greatly improve bandwidth utilization and thus effective upload speed on larger files. Do be aware that this will make rclone use NumberOfTransfers(default4) x 128MB extra RAM so make sure you have enough. if not, use a lower number)

The full command then would look something like this:

rclone move /home/username/media/Movies gdrive:movies --fast-list --progress --drive-chunk-size 128M

That should cover the basics and give you close to ideal performance.
When transferring lots of small files (which gdrive is inefficient at), --transfers 8 can also be useful to speed that up a little, but I assume for your movies this is a non-issue (and it will double the memory cost of that 128M chunk size setting) so I'm not including it in this spesific recommendation.

Thank you for all your help again .

So when I create the cache in rclone. When it asks for remote path do I set the path to the location I create IE: username/plexdrive or create folder in day /mnt/gdrive then point gdrive and cache to that folder.

Can I use cache without encryption to ??

Thanks

No this is wrong, see my example below on how to do it.

[TD1]
type = drive
client_id = [REDACTED]
client_secret = [REDACTED]
scope = drive
token = [REDACTED]

^ TD1Cache points to TD1

[TD1Cache]
type = cache
remote = TD1:

^ TD1CacheCrypt point to TD1Cache

[TD1CacheCrypt]
type = crypt
remote = TD1Cache:
filename_encryption = standard
directory_name_encryption = true
password = [REDACTED]
password2 = [REDACTED]

I highlighted the important bit for you in terms of how to link them to work together.
Note that when you used this system (with both cache and crypt) you want to use TD1CacheCrypt when you mount for example. If you just mounted TD1 you'd just get the regular access with neither crypt nor cache. You'd be shortcutting everything.

The key here is what you set "remote" line to be - as this is where the data goes as the next step. You want each link in the chain to point to the name of the next such that you get Crypt --> Cache --> Gdrive

And yes, you can of course run cache without crypt, or crypt without cache.
In this example config if I wanted to skip the encryption I could just delete that whole remote and then start using TD1Cache directly instead. Technically it's not even necessary to delete the crypt remote as it would't do anything on it's own if you just shortcut past it.
If I wanted to use crypt without cache I'd delete the cache remote and then point the crypt remote directly to TD1 (because it used to point to the Cache remote and that doesn't exist anymore so it would give an error).

Hopefully that makes it clearer? :smiley:

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.