Huge difference in file count with dry-run on and off?

Hey all, quite new to rclone, so apologies if I’ve missed something very obvious here!

I’m running this command:

rclone --exclude "@eaDir/**" --exclude "@eaDir/" sync -v --config="/volume1/homes/xnaas/.config/rclone/rclone.conf" /volume1/Media gcrypt: --log-file=/volume1/homes/xnaas/logs/rclone.log

This is to do my initial backup to my Google Drive account. It’s backing up stuff encrypted without any issues, but I noticed the file count was very low (only ~10000). If I modify the command and add --dry-run, it does actually see all ~70000 files. This backup is over 10TB of stuff. When it only sees the ~10k files, that’s not even 2TB.

Any thoughts on what’s happening here? :thinking:

Never mind. After it’s been running for awhile, I see that more files are showing up. We’re up to ~14k now, so I hope it eventually sees everything as it continues to progress through uploads. :slight_smile:

It should do.

By default the reading process can only get 10k files ahead of the transferring process so it doesn’t fill your memory up. You can change this with

  --max-backlog int                          Maximum number of objects in sync or check backlog. (default 10000)

If you want.

1 Like

Aha! Sweet, thanks for the info!

1 Like