We are attempting to solve a problem. We are using rclone.exe sync command with an API from our customers vendor swiftstack. We just upgraded the server on the backend storage to SSD drives, and we have 24 CPU’s and 64GB Of ram on a virtual server.
We have over 50TB of data with hundreds of millions of files and directories. I know we have a few bottlenecks with directory checking and validation between the local drive and the remote swiftstack appliance. At the current time we are running a version of 1.42 from the vendor. At the current time per the screenshot below you can see that the server is using C:\pagefile.sys to process and facilitate the transfers and not utilizing the physical ram. This is a VM on a new infrastructure.
IS there a way to have rclone.exe sync bypass loading into the pagefile.sys? Anyone have any suggestions? We could turn off the pagefile.sys though I think just letting windows manage it would be a better options.
Screenshot of server load on disk and pagefile.sys being used with rclone.exe