Journaling? sync or copy with less disk space


#1

So i am copying a disk which is larger than mine to my local hard drive. By definition i cannot copy/sync everything to my hard drive because the destination drive is smaller than the source drive.
I am now copying everything in the background and moving the copied files to external drives in the foreground maintaining some free space on the destination drive. However, when i need to restart the copy/sync operation for whatever reason (connection lost, for example) this workflow fails because copy and sync want to make the two directories identical.
Is there a way to keep some sort of a journal of all files that were copied successfully and not trying to copy them again, no matter if they exist on the destination or not?


#2

Hi @Toontje,

I’m not sure if this is possible with rclone. however, you can use Filtering or more specifically --exclude-from which can read a list of files from a txt file and exclude them from the backup/sync.

So you would need to find a way of getting rclone to add the file path to the txt file after it has successfully copied it. this is the part which I’m not sure if rclone can do,

I’m sure someone else can shed some light on this, or may know another way this can be done.