Rclone sync filling tmp, kills system

Help and Support
1

Hi,

I'm using rclone to sync files to Jottacloud and have noticed that when syncing files, rclone puts them in tmp first. Now I'm transferring several large files which results in tmp filling up and since other services are dependant on the drive where tmp resides, these services start to crash.

Took me a good while to even find these tmp files since they don't show up in any other way except with:
lsof -s | grep 'deleted
rclone 17423 root 23u REG 0,44 206G 49050 /mnt/SSD/tmp/rclone-jcmd5-692059085 (deleted)
rclone 17423 root 24u REG 0,44 75G 48731 /mnt/SSD/tmp/rclone-jcmd5-613371592 (deleted)
rclone 17423 root 25u REG 0,44 115G 48352 /mnt/SSD/tmp/rclone-jcmd5-715811207 (deleted)

These files will no doubt disappear when the sync finishes and the rclone sync command exits but syncing 400GB of files will take a good while and seeing as the system becomes pretty much unusable during that time and I have to kill the sync, I'm pretty much in a catch 22.

My sync command is:
rclone sync --exclude stuff/** --exclude "/other stuff/**" --log-file $LOG_PATH/jc_sync_$DATE.log --log-level INFO --modify-window 3601s --transfers 6 /mnt/HDD/Storage/Backup jc-crypt:/Backup

Is there something I can do to prevent this from happening?

Br,
Chronic

2

What does rclone version show?

3

rclone v1.49.5

  • os/arch: linux/amd64
  • go version: go1.12.10
4

See:

5

Ah, SOL in other words. :frowning:

6

No, use the variable to point the temp file somewhere with space. It's written up in the issue.

7

There is no other place with more space, just like in the issue.

8

You can reduce the transfers and also you have no space to store a single file of the largest size?

9

These files should disappear once the file has been uploaded.

Is that what you see, or do you see the number of deleted files build up?

10

Yes, they disappear but that's not the issue here. :slight_smile:

11

Good, just checking there isn't a bug!

Unfortunately you can't upload files to Jottacloud without knowing the MD5 in advance. Because you are using crypt, rclone has to encrypt the entire file, then calculate the MD5 before uploading - hence the need for temporary space.

12

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.