Measure rclone process network use on linux

What is the problem you are having with rclone?

not really a problem, i would like to measure the total network/internet usage of my weekly rclone -sync task.

I tried several tools, top, nethogs, iftop, and googled a lot, but cannot find any way to record the total traffic rclone creates for a complete sync run. I assume rclone needs to transfer data beetween the remote and local at least to compare the data, so the statistics output for "Transferred:" is understood as the new/changed data, but this is assumed not to contain additional network bandwidth for the rclone checks, etc.

I have around 300 GB of data, synced weekly to googleDrive and Mega clouds, and a sync transfers around 2GB. But i want to find out what the total network/internet usage is as I sit sometimes with a 5G SIM connection to the net and the network usage counts.

Any hint on a tool that can measure the send / received of rclone during the sync would be excellent.

Run the command 'rclone version' and share the full output of the command.

rclone v1.66.0

  • os/version: ubuntu 24.04 (64 bit)
  • os/kernel: 6.8.0-35-generic (x86_64)
  • os/type: linux
  • os/arch: amd64
  • go/version: go1.22.1
  • go/linking: static
  • go/tags: none

Which cloud storage system are you using? (eg Google Drive)

googleDrive and Mega

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone sync localDir encrypted-union-remote: --progress --links --create-empty-src-dirs --delete-during --delete-excluded --human-readable --log-file=~/log_it_baby.log

Please run 'rclone config redacted' and share the full output. If you get command not found, please make sure to update rclone.

-- can be sent on request --

A log from the command that you were trying to run with the -vv flag

-- if required -- 

This is not really something you can get from rclone itself - I do not think anybody planned for it.

Look at your OS specific tools - for Linux try for example nethogs or dstat. Both allow to measure cumulative network traffic for specific process.

1 Like

Thanks, as mentioned, i tried several tools, but failed. nethogs sums up the bandwith for an interval, but does not record the sum total. I never asked for this to be done by rclone, a tool that can do that would be my goal to find.

There is other very simple way:) I am sure your mobile device records your traffic - if not device then your operator for billing purposes (otherwise I guess you would not worry about few GB on 5G network).

Make sure nothing else is using network. Get initial stats. run rclone sync. Get stats. Do math. Done.

thanks for trying. I seek a TOOL to record incoming and outgoing traffic of the rclone process on ubuntu and sum up the totals. if you know such a tool welcome. thanks.

Try on ubuntu forums. This is general linux problem.

thanks, have you ever heard that your comments might be pretty useless and embarrassing? I am sure if you would sit on the "ubuntu forums" you refer to you would write "this is a rclone problem, try on rclone forums". Does not make sense sir. Get a grip on what you write!

It is not rclone problem. You are asking how to measure network interface traffic for specific process on your OS:) If you ask your questions in the right place you might have some meaningful answers.

well, thanks fro your opinion. my view differs, as neither the forum nor the question are the problem, but your repeated empty response.

I have found the solution now.

One needs 2 shells / terminals.

Terminal 1:
start rclone: rclone sync localDir remote: -your flags and parameters

Terminal 2:
start nethogs: sudo nethogs -P pgrep rclone -v 3

pgrep gets the pid of the rclone process (if you have more rclone processes running you need another way to get the pids and feed it to nethogs via -P)

-v 3 tells nethogs to sum the total (this is what i missed). The default is -v 0 which lists the bandwidth.

After the rclone job terminates you can see in terminal 1 how much files were updated (the "Transferred" data), and in terminal 2 you see the real network usage, e.g. "Transferred" data + rclone protocol overheads.

So far as it should be, proof to be made, and is already running.

look at my post hope it helps you.

thanks, thats very impressive - great job!

First run's results - 262.667 GiB archive on a googleDrive encrypted union, synchronised against the local folder.

982 MiB were changed and thus transferred from local to googleDrive.

nethogs recorded 3.151 GB sent and 2.16 GB received, so i assume one sync round takes ~5.5 GB (i am not sure how billing/accounting is done at the network provider, whether its the sum of up/download).

gd_stat