Using RClone for file transfers on a Jetson Xavier cause system to shutdown

What is the problem you are having with RClone?

So, I'm using RClone to backup data from my edge devices, which happens to be Nvidia Jetson Xavier ARM-based devices. Whenever I try to copy a directory with many sub-directories and overall size of approx 31 GB the whole system shuts down.
Is there any system-level or RClone's own logs that can help me figure out this issue?

What is your rclone version (output from rclone version)

rclone v1.53.3

  • os/arch: linux/arm64
  • go version: go1.15.5

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Ubuntu 18.04

Which cloud storage system are you using? (eg Google Drive)

SFTP

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone copyto /mount/ssd/my_data sftp_remote:/home/backups/my_data

The rclone config contents with secrets removed.

type = sftp
host = xxx.xxx.xxx
user = xx
port = 22
pass = xxxxxxx
key_use_agent = false
use_insecure_cipher = false
disable_hashcheck = true

A log from the command with the -vv flag

The system shuts down, can't view logs

Probably running out of memory, try to limit transfers or checkers to 1 and 1 and see how things go.

Thanks for replying so quickly. I was monitoring Htop, and looking at that, the memory never went up more than 2GB out of a total of 8GB, similarly, CPU usage was pretty much around 20%.

Either way, I'll try your suggestions and come back. Just to be sure you recommend adjusting --transfers=N and --checkers=N, right?

I'll try something like this, basically halving the defaults:

rclone copyto --transfers=2  --checkers=4 /mount/ssd/my_data sftp_remote:/home/backups/my_data

Rclone doesn't shut the system down.

Did it crash? Power off?

A hard crash that just ends up with the system down. Have to manually turn it back on.

If the system is crashing, there isn't much a rclone log would tell as you'd have to figure out how to get a crash log on the system and see what's going on.

That does seem really strange though.

It sounds like a hardware problem. It could be any of these

  • Bug in kernel
  • Bug in driver (eg network driver)
  • Hardware problem (eg overheating)
  • RAM problem (bad RAM)

Those are the most common ones I think.

Is there anything in the system log (dmesg)?

You might have to get a serial console on it to see what is happening.

@Animosity022 So, seems like reducing transfers and checkers to 2 and 4, respectively, did the trick. Though I have to agree with @ncw this is probably a bug that will need to be diagnosed further through the serial console, since due to the abrupt nature of the system-wide crash there isn't enough time to store any sys-logs.

I did come across a similar rclone issue on ask Ubuntu (for some reason can't post a link) where the answer recommended running Passmark Memtest86, but since it is an x86 based utility I'll have to dig a bit deeper to run a memory test on ARM

That is probably indicative of one of the problems above.

Rclone is highly multhreaded and can work CPU/RAM/network very hard! It has crashed a lot of home routers in the past :wink: (which were usually fixed with a firmware upgrade).

You could try one of my other programs stressdisk which checks disks but incidentally detects bad RAM too.

1 Like

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.