How to check mount upload progress from scripts?

Help and Support
1

What is the problem you are having with rclone?

Hi,
how can I check rclone mount upload progress from other scripts? I want to use a combination of Borg Backup and rclone mount like this:

  • Mount cloud storage with rclone mount
  • Run Borg Backup --> Files will be in the VFS Cache
  • Wait until all files have been uploaded from the VFS Cache to the cloud storage
  • Unmount cloud storage

Of course I could parse and check the log file for these messages:
vfs cache: cleaned: objects 0 (was 2) in use 0, to upload 0, uploading 0, total size 0 (was 76.831Mi)

But I'm not sure whether this is the best solution. It would be cool if I could get these values to check the progress with rclone rc, is that possible?

What is your rclone version (output from rclone version)

1.57.0

Which cloud storage system are you using? (eg Google Drive)

At the moment WebDAV for testing

The command you were trying to run (eg rclone copy /tmp remote:tmp)

./rclone mount --rc --vfs-cache-mode writes --log-level INFO --cache-dir /daten/rclone_vfs_cache --vfs-write-back 60s --vfs-cache-max-age 10m --vfs-cache-max-size 10G --stats 1m  --stats-one-line Bitrix24_Test:Backup/ /cloud/bitrix24/
2

If you are using writes, having the mount in the middle is an unneeded and spot for problems.

I think you'd be better off writing to a temp location and using rclone move from that temp location. You can monitor the job and once the job completes, you are done. Much easier to script as well.

3

I'm afraid this will not work with Borg Backup as Borg needs no temp location but the proper backup repository. Of course it would work for the first backup but not for the following ones. Therefore mount seems to be the only solution (besides local repo synced to cloud storage which is not what I want).

4

See:

5

Hi again :slight_smile:

The lastest replies in #3641 look quite positive and my first tests with Borg worked really well. Now I just need to find out when syncing has completed without searching through the logfiles.

6

Check out:

https://rclone.org/rc/#core-stats

7

Hi,
sorry for my late response due to a business trip.

The core stats look quite interesting, but I am not sure how to evaluate the given information for my use case. Basically they tell me the status of transfers, but not of the VFS Cache. Is there something similar for the VFS Cache?

Like for example:

rclone rc vfscache/stats
{
  "objects": 10,
  "inUse": 5,
  "toUpload": 3,
  "uploading": 2,
  "totalSize": 131072
}

That would be awesome!

8

I don't think that exists unless there is something not documented yet. @ncw can most likely confirm.

I'd just parse the log file personally as all the info and what you want is there. Not sure that's much different from parsing the RC command.

9

I agree that would be awesome, but it doesn't exist at the moment :frowning:

Can you open a new issue on Github about this as I think it would be useful thing to add.

10

Hi ncw,

great to read that you think this would make sense. I just opened Query rclone mount VFS stats via rclone remote control API · Issue #5816 · rclone/rclone · GitHub

Best regards,
jdr85

1 Like
11

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.