Ncdu not working if --fast-list is used

[xd003 (Badal Singh)]

What is the problem you are having with rclone?

I am just trying to explore my remote using ncdu but it doesn't seem to work . The ncdu is just stuck on waiting for root mentioned at top

What is your rclone version (output from rclone version)

rclone v1.54.0

• os/arch: linux/amd64
• go version: go1.15.7

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 20

Which cloud storage system are you using? (eg Google Drive)

Google drive

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone --config=$HOME/easyclone/rc.conf ncdu src: --fast-list

The rclone config contents with secrets removed.

[src]
type = drive
client_id = xxx
client_secret = xxx
scope = drive
root_folder_id = xxx
service_account_file = $HOME/easyclone/accounts/1.json

A log from the command with the -vv flag

2021/02/09 16:48:41 DEBUG : rclone: Version "v1.54.0" starting with parameters ["rclone" "--config=/home/ubuntu/easyclone/rc.conf" "ncdu" "src:" "--fast-list" "-vv"]
2021/02/09 16:48:41 DEBUG : Using config file from "/home/ubuntu/easyclone/rc.conf"
2021/02/09 16:48:41 DEBUG : Creating backend with remote "src:"
2021/02/09 16:49:24 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=1054028452057, userRateLimitExceeded)
2021/02/09 16:49:24 DEBUG : pacer: Rate limited, increasing sleep to 1.156985839s
2021/02/09 16:49:24 DEBUG : pacer: Reducing sleep to 0s
2021/02/09 16:49:27 DEBUG : pacer: low level retry 1/10 (error googleapi: Error 403: User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=1054028452057, userRateLimitExceeded)

If I don't use --fast-list flag ,even then i do these in logs but the ncdu doesn't gets stuck at all , it works totally fine!

Attached below is the screenshot showing ncdu getting stuck when --fast-list is used

[asdffdsa (jojothehumanmonkey)]

hello,

for me, ncdu is much faster using --fast-list.
rclone ncdu wasabi01: --fast-list

seems that using --fast-list, rclone does not display progress while calculating.

[xd003 (Badal Singh)]

@asdffdsa
Do you mean mean to say when ncdu is used along with --fast-list , its normal for it to not display any progress but it is actually working in background and will only display final results once its fully caclulated ?

[Animosity022]

Fast-list is grabbing all your items because you've asked to recursively get it all.

You would have to wait until the full sweep is done before anything would be displayed. If you have a bigger items/drive, it would take longer.

Without fast-list, it steps down the paths and is slower overall, but you'd get results back sooner on your display even though they are not complete.

[asdffdsa (jojothehumanmonkey)]

yes,

@Animosity022 gave a good answer.