So, I'm using RClone to backup data from my edge devices, which happens to be Nvidia Jetson Xavier ARM-based devices. Whenever I try to copy a directory with many sub-directories and overall size of approx 31 GB the whole system shuts down.
Is there any system-level or RClone's own logs that can help me figure out this issue?
What is your rclone version (output from rclone version)
rclone v1.53.3
os/arch: linux/arm64
go version: go1.15.5
Which OS you are using and how many bits (eg Windows 7, 64 bit)
Ubuntu 18.04
Which cloud storage system are you using? (eg Google Drive)
SFTP
The command you were trying to run (eg rclone copy /tmp remote:tmp)
Thanks for replying so quickly. I was monitoring Htop, and looking at that, the memory never went up more than 2GB out of a total of 8GB, similarly, CPU usage was pretty much around 20%.
Either way, I'll try your suggestions and come back. Just to be sure you recommend adjusting --transfers=N and --checkers=N, right?
I'll try something like this, basically halving the defaults:
If the system is crashing, there isn't much a rclone log would tell as you'd have to figure out how to get a crash log on the system and see what's going on.
@Animosity022 So, seems like reducing transfers and checkers to 2 and 4, respectively, did the trick. Though I have to agree with @ncw this is probably a bug that will need to be diagnosed further through the serial console, since due to the abrupt nature of the system-wide crash there isn't enough time to store any sys-logs.
I did come across a similar rclone issue on ask Ubuntu (for some reason can't post a link) where the answer recommended running Passmark Memtest86, but since it is an x86 based utility I'll have to dig a bit deeper to run a memory test on ARM
That is probably indicative of one of the problems above.
Rclone is highly multhreaded and can work CPU/RAM/network very hard! It has crashed a lot of home routers in the past (which were usually fixed with a firmware upgrade).
You could try one of my other programs stressdisk which checks disks but incidentally detects bad RAM too.