Best practices for Mounting GameServer cache files to save space

Help and Support
1

What is the problem you are having with rclone?

No problem with Rclone itself, it does its job well. I am just wondering if there are any improvements I may have overlooked, as I'm still fairly new to using it.

My game server uses a ton of large custom maps from the Steam Workshop, the thing is we only play a single map at a time, but like having a large variety. That results in the cache folder with all of those maps that the server has downloaded from the workshop ballooning up to 3x the size of the core server files themselves. The group realizes that some added loading time is unavoidable, and are fine with that, I'm just hoping for maybe some tips on how to cut that down. (The reason it's necessary in the first place, is to fit our server onto a new performance VPS that we would not be able to utilize otherwise, and the community decided that lower latency, and the overall increase in performance during games was worth the additional loading time.)

What is your rclone version (output from rclone version)

v1.57.0 (Latest at the time of writing)

Which cloud storage system are you using? (eg Google Drive)

Wasabi S3 compatible bucket. uc-central-1

The command you were trying to run (eg rclone copy /tmp remote:tmp)

I should add that I have /mount/ Symlinked to the Cache directory the server downloads it's maps to

rclone mount --allow-other --vfs-cache-mode writes wasabi:/bucketname/Cache /home/user/mount &

The rclone config contents with secrets removed.

[wasabi]
type = s3
provider = Wasabi
access_key_id = ################
secret_access_key = #######################
region = us-central-1
endpoint = s3.us-central-1.wasabisys.com
location_constraint = s3.us-central-1.wasabisys.com
acl = private

Thank you in advance for any advice or wisdom the community may impart!!

2

If you simply want to reduce the size of your cache you can use the vfs-cache-max-size flag.

3

hello and welcome to the forum,

--vfs-cache-mode writes, as i understand it, "files opened for read only are still read directly from the remote".
so if there are files in the cache, would i be correct that steam is modifying them?

4

So increasing rclone's cache would decrease my server cache? That's a pretty good thing to know for a separate
seedbox project, thank you!

But in this case, the size and space issue has already been solved for the most part. I'm hoping to cut down on the loading times when the server switches maps, having to fetch a copy of the data has basically doubled the load times.

To the best of my understanding, the server/steam run a quick version check while a player either joins, or the server switches maps, seeing if the workshop file that a player has on their local machine matches the version that the server has. But I don't think steam is actively updating the map or anything while it's in progress. But the map does need to load up onto the server for it to be played.

In my perfect world, all the map files would live solely on Wasabi, and just the map that's about to be played would get cached, one or two maps at a time. I saw someone's post about a script that cache's the next episode for Plex, I wonder if something like that could be done here, grabbing the next map in advance so that it's ready before the server actually switches to it..

5

if rclone is not caching the reads and steam is not modifying the maps, then what is in the cache?
perhaps steam is opening the files as read/write and thus forcing rclone to cache them.

my local emby server uses a rclone mount without any vfs cache.

  • not using --vfs-cache-mode. in effect --vfs-cache-mode=off
  • using --read-only

perhaps try that and see what happens

6

Forgive me, I think I wasn't particular enough in my initial explanation. The Cache that I've been battling with is NOT the Rclone cache. It is the cache of Steam Workshop content on my actual Game server.

And it was that "Cache" that I have mounted off-site. And now I'm trying to narrow down the most efficient and quickest way to set things up to minimize the load times as Rclone caches the Game Server Cache.

Edit: I will absolutely be trying it momentarily with read-only. I wasn't even aware that would be an option for my case. Thank you!

7

ok, in that case, might test using --vfs-cache-mode=full.
that downloads just the data that steam requests.

for example, with an emby scan, let's say emby finds a new 50GB media file in my rclone mount.
emby will need to scan/read that file for metadata, to read, let's say 1MB.
so rclone will download in the rclone vfs cache, just that 1MB of the media file, not the entire 50GB.

of course, if steam scans/reads the entire map file, then rclone would have to download the entire file into the vfs cache.

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.