Work Flow Thoughts

[Animosity022]

I had someone comment here about an issue with a rclone move:

animosity22/homescripts

Finally figured out that the user moved from a mounted remote to the same remote, which at the end, deletes the file.

Everything works as it should in theory as I would surmise rclone doesn't know the remote is a mounted remote basically deletes it.

felix@gemini:/gmedia$ rclone move /gmedia/hosts gcrypt: -vv
2020/06/15 13:40:00 DEBUG : rclone: Version "v1.52.1" starting with parameters ["rclone" "move" "/gmedia/hosts" "gcrypt:" "-vv"]
2020/06/15 13:40:00 DEBUG : Using config file from "/opt/rclone/rclone.conf"
2020/06/15 13:40:00 DEBUG : fs cache: renaming cache item "/gmedia/hosts" to be canonical "/gmedia"
2020/06/15 13:40:01 DEBUG : hosts: Size and modification time the same (differ by 0s, within tolerance 1ms)
2020/06/15 13:40:01 DEBUG : hosts: Unchanged skipping
2020/06/15 13:40:01 INFO  : hosts: Deleted
2020/06/15 13:40:01 INFO  :
Transferred:   	         0 / 0 Bytes, -, 0 Bytes/s, ETA -
Checks:                 2 / 2, 100%
Deleted:                1
Elapsed time:         0.5s

2020/06/15 13:40:01 DEBUG : 6 go routines active

I can put in my upload script to check the source isn't a fuse mounted fs, but not sure what else I could do to prevent something like that.

Any other thoughts or suggestions would be welcome.

[Lex]

@Animosity022

Finally figured out that the user moved from a mounted remote to the same remote, which at the end, deletes the file.

Aside from telling the user: "Don't move from A: to A:" ...

Interesting logic challenge. Perhaps you could send a zero-byte file (similar to mounted.bin) with a random name 4723sdf080hhev897w4thihs.bin to remote: then do a find or rclone ls on the local to see if a) the file exists and optionally b) the time stamp is identical. Then delete the random bin file.

Ugly but ...

[ncw (Nick Craig-Wood)]

@Animosity022

Finally figured out that the user moved from a mounted remote to the same remote, which at the end, deletes the file

Foo!

Rclone will detect this if you try rclone move remote: remote: but it is really difficult to check in the general case - you'd have to look in /proc/mounts for an rclone mount - that will tell you if the mount point "/gmedia/hosts" is under "gcrypt:" or not

$ rclone mount TestDrive: /mnt/tmp

$ grep rclone /proc/mounts 
TestDrive: /mnt/tmp fuse.rclone rw,nosuid,nodev,relatime,user_id=1000,group_id=1000 0 0

Do you think rclone should be doing this?

[ncw (Nick Craig-Wood)]

There is always this flag as a safety

  --max-delete int   When synchronizing, limit the number of deletes (default -1)

[Animosity022]

Hmm, that's not a bad idea.

I could make it something large I guess as I am using move on purpose as I want to move from local to cloud.

I also put a simple check to make sure it's not a fuse mount that it's moving from so that should help as well:

#is $LOCAL actually a local disk?
if /bin/findmnt $LOCAL -o FSTYPE -n | grep fuse; then
	echo "fuse file system, exitting"
	exit 1
fi

Dirty, but does the job!

[ncw (Nick Craig-Wood)]

I didn't know about findmnt!

Note that it only seems to show stuff if $LOCAL is the root of the mount

$ findmnt / -o FSTYPE -n
ext4
$ findmnt /bin -o FSTYPE -n
$

[Animosity022]

Sorry as I have $LOCAL configured up above so if they change the environment variable, it works through the script.

So a bit higher, I have:

# Local Drive
# This must be a local mount point on your server that is used for the source of files
# WARNING: If you make this your rclone Google Drive mount, it will create a move loop
# and delete your files!
# Make sure to this to the local path you are moving from!
LOCAL=/local

For me, it's just /local.