Enhancements for core/stats

[negative0 (Chaitanya Bankanhal)]

1. Improving the stats which are returned by core/stats to incorporate all the information about the currently running transfers. Currently, we have to manually map the jobs in core/stats's transferring to core/transferred, and we don't return anything for jobs that are queued to be started, we should be able to easily get currently waiting, transferring and transferred jobs in separate arrays, and they can be mapped using a group id and a file name.

2. Also, when we try to upload a folder, I think we should pre-group the stats and nest the individual jobs inside the original job, and provide a sum of details of the group, as well as the individual job.

3. The core/stats should return the ingress and egress that is being done by rclone.

How do you think rclone should be changed to solve that?

We should return everything we have about current transfers in the core/stats itself,

• Current upload speed
• Current download speed
• Currently transferring
• Checking/ Queued
• Renaming
• Deleting
• Transferred

[ncw (Nick Craig-Wood)]

@negative0

Currently, we have to manually map the jobs in core/stats's transferring to core/transferred,

I'm not sure what you mean here? The "transferring" jobs are in progress the "transferred" jobs are finished.

@negative0

and we don't return anything for jobs that are queued to be started

Strictly speaking they aren't jobs yet. I suspect you mean the transfer queue? The overall numbers for this are supposed to be in core/stats but they don't appear to be working...

They are supposed to be here in totalTransfers

totalChecks: 0,
totalTransfers: 15,
transferTime: 0,

But the stats from rclone look like this

Transferred:   	  727.110Ki / 1.013 GiByte, 0%, 409 Byte/s, ETA 4w2d16h42m22s
Transferred:           12 / 5667, 0%
Elapsed time:      11m0.0s

That 5667 number is nowhere to be seen, so something is wrong here.

@negative0

we should be able to easily get currently waiting, transferring and transferred jobs in separate arrays, and they can be mapped using a group id and a file name.

"transferring" are in core/stats and "transferred" are in core/transferred

However "waiting" isn't available yet.Do you want a list of these file names? There can be 10,000 names in the to-be-transferred queue which I think might be too many.

@negative0

Also, when we try to upload a folder, I think we should pre-group the stats and nest the individual jobs inside the original job, and provide a sum of details of the group, as well as the individual job.

That sounds complicated! What are you trying to achieve here? What would you like to be displaying to the user?

@negative0

The core/stats should return the ingress and egress that is being done by rclone.

That would be nice, I agree. Currently we return the throughput through rclone only.

Network ingress and egress would be possible to measure for most backends - I think that is a great idea.

So we should probably make issues about

• totalTransfers not working
• adding network ingress/egress to core/stats

@negative0

We should return everything we have about current transfers in the core/stats itself,

• Current upload speed
• Current download speed
• Currently transferring
• Checking/ Queued
• Renaming
• Deleting
• Transferred

Apart from upload and download speed, the others should be in the stats already (at least the total numbers for the queues).