not really a problem, i would like to measure the total network/internet usage of my weekly rclone -sync task.
I tried several tools, top, nethogs, iftop, and googled a lot, but cannot find any way to record the total traffic rclone creates for a complete sync run. I assume rclone needs to transfer data beetween the remote and local at least to compare the data, so the statistics output for "Transferred:" is understood as the new/changed data, but this is assumed not to contain additional network bandwidth for the rclone checks, etc.
I have around 300 GB of data, synced weekly to googleDrive and Mega clouds, and a sync transfers around 2GB. But i want to find out what the total network/internet usage is as I sit sometimes with a 5G SIM connection to the net and the network usage counts.
Any hint on a tool that can measure the send / received of rclone during the sync would be excellent.
Run the command 'rclone version' and share the full output of the command.
rclone v1.66.0
os/version: ubuntu 24.04 (64 bit)
os/kernel: 6.8.0-35-generic (x86_64)
os/type: linux
os/arch: amd64
go/version: go1.22.1
go/linking: static
go/tags: none
Which cloud storage system are you using? (eg Google Drive)
googleDrive and Mega
The command you were trying to run (eg rclone copy /tmp remote:tmp)
Thanks, as mentioned, i tried several tools, but failed. nethogs sums up the bandwith for an interval, but does not record the sum total. I never asked for this to be done by rclone, a tool that can do that would be my goal to find.
There is other very simple way:) I am sure your mobile device records your traffic - if not device then your operator for billing purposes (otherwise I guess you would not worry about few GB on 5G network).
Make sure nothing else is using network. Get initial stats. run rclone sync. Get stats. Do math. Done.
thanks for trying. I seek a TOOL to record incoming and outgoing traffic of the rclone process on ubuntu and sum up the totals. if you know such a tool welcome. thanks.
thanks, have you ever heard that your comments might be pretty useless and embarrassing? I am sure if you would sit on the "ubuntu forums" you refer to you would write "this is a rclone problem, try on rclone forums". Does not make sense sir. Get a grip on what you write!
It is not rclone problem. You are asking how to measure network interface traffic for specific process on your OS:) If you ask your questions in the right place you might have some meaningful answers.
pgrep gets the pid of the rclone process (if you have more rclone processes running you need another way to get the pids and feed it to nethogs via -P)
-v 3 tells nethogs to sum the total (this is what i missed). The default is -v 0 which lists the bandwidth.
After the rclone job terminates you can see in terminal 1 how much files were updated (the "Transferred" data), and in terminal 2 you see the real network usage, e.g. "Transferred" data + rclone protocol overheads.
So far as it should be, proof to be made, and is already running.
First run's results - 262.667 GiB archive on a googleDrive encrypted union, synchronised against the local folder.
982 MiB were changed and thus transferred from local to googleDrive.
nethogs recorded 3.151 GB sent and 2.16 GB received, so i assume one sync round takes ~5.5 GB (i am not sure how billing/accounting is done at the network provider, whether its the sum of up/download).