Order of statistics

[Max-Sum]

Problem

• Order of statistics doesn't align with the starting time of a transfer.
• Unstable output on WebUI

Proposal

Change stringSet to use linkedHashMap.

Besides, anyone know why the speed on WebUI skyrocket when transferring some small files?

[ncw (Nick Craig-Wood)]

@Max-Sum

Order of statistics doesn't align with the starting time of a transfer.

The stats are sorted by name.

Are you suggesting we sort them by start time instead? Or maybe as well?

@Max-Sum

Change stringSet to use linkedHashMap.

I don't think we need to do that, just change the sort if we decide that is what we want to do.

@Max-Sum

Unstable output on WebUI

These are being transferred to the WebUI via JSON so all order will be lost there I suspect. I think it will be up to the WebUI to order them correctly.

@Max-Sum

Besides, anyone know why the speed on WebUI skyrocket when transferring some small files?

The webui just shows data rclone sends it so you should be able to see this in rclone too I would have thought.

[Max-Sum]

@ncw

The stats are sorted by name.

Are you suggesting we sort them by start time instead? Or maybe as well?

I would suggest sorting by start time. New transfers at the bottom make more sence to me.

@ncw

I don't think we need to do that, just change the sort if we decide that is what we want to do.

I saw the implementation of statistics. If a transfer is not yet started, progress.get(name) could be nil (stringset.go:82) and thus we have no way to the start time.

@ncw

The webui just shows data rclone sends it so you should be able to see this in rclone too I would have thought.

I didn't try in rclone. I suspect the transfer time of small files are underestimated.

[Max-Sum]

@ncw

These are being transferred to the WebUI via JSON so all order will be lost there I suspect. I think it will be up to the WebUI to order them correctly.

This is because the JSON output is not sorted like the normal statistics output.

[Max-Sum]

@Max-Sum

I saw the implementation of statistics. If a transfer is not yet started, progress.get(name) could be nil (stringset.go:82) and thus we have no way to the start time.

I was wrong. *Transfer contains a startTime so that it could be used. In such case, extending stringSet into transferMap (map[string]*Transfer) could be even better.
See: https://github.com/Max-Sum/rclone/tree/sortByStartTime

The changes could:

• Maintain O(1) for adding / deleting / looking up
• Allow easier access to transfer info

[ncw (Nick Craig-Wood)]

I looked at your code - looks good!

I think the only actual change is here other than renaming/moving files?

	sort.Slice(s, func(i, j int) bool {
		return s[i].startedAt.Before(s[j].startedAt)
	})

Can you send a PR with two commits in, one doing all the renaming and moving of files (which I think is a great idea!) and the next with the actual change?

Thank you - I think this will be a nice change.