Trying to transfer over 6TB of data via sftp. Transfer interrupt regularly after an irregular amount of data transferred with an exit code of 2. The transfer ends between 30 and 500GB.
What is your rclone version (output from rclone version)
rclone v1.50.2
os/arch: linux/amd64
go version: go1.13.4
Which OS you are using and how many bits (eg Windows 7, 64 bit)
openSUSE 13.1 (Bottle) (x86_64)
Which cloud storage system are you using? (eg Google Drive)
sftp
The command you were trying to run (eg rclone copy /tmp remote:tmp)
A log from the command with the -vv flag (eg output from rclone -vv copy /tmp remote:tmp)
-vv doesn't work for whatever reason, the command does nothing and comes back directly to the command line prompt.
Grepping for fatal in the log shows the following from multiple interruptions:
fatal error: sweep increased allocation count
fatal error: acquireSudog: found s.elem != nil in cache
fatal error: fault
fatal error: workbuf is empty
fatal error: workbuf is not empty
fatal error: workbuf is not empty
fatal error: workbuf is not empty
fatal error: workbuf is not empty
fatal error: workbuf is not empty
I can provide more log, please let me know which part, since it is huge and has needs lots of cleaning.
Those are strange errors! I looked some of them up and they all seemed to come from very old go versions - however your rclone version above says you are using a new version. Can you double check you are using the v1.50.2 version in the script?
If you would like to share it with me privately I'll take a look. Either PM me a link or email to nick@craig-wood.com with a link to this forum page.
Thank you very much for looking into this. I sent you the log file. rclone is current, I downloaded it from the rclone page, but the OS is old (openSuse 13.1). Could this be a problem?
That is a very strange log! All the errors are from the go runtime and when I search them there are no current problems I can see with those symptoms. It is odd that there are so many different types of error. Some of the stack traces look corrupted and there are a few SIGSEGVs in there too which is very odd.
This makes me think something else is up with your machine.
I think you are probably running a kernel like 3.11 (uname -a will show). The go runtime should work with anything older than 2.6.23 so this should be new enough....
Can you try a newer kernel?
The cause looks like memory corruption of some kind. rclone doesn't use any cgo (interface with C modules) so there shouldn't be any opportunity for rclone to corrupt its own memory.
The fact that there are so many different fatal errors makes me wonder if there is a memory problem with your machine. Is it normally reliable? Have you tried memtest86 on it?