Union mount and hardlinking/moving instead of copying files

Help and Support
1

Thanks for the Union refactor, there are some great new features there.

I have been using it for fairly simple purposes:

[union]
type = union
upstreams = /data/upload encgoogle::ro

I then mount it to a folder on the local drive via the official docker container. The command equates to the following:

rclone:
    cap_add:
        - SYS_ADMIN
    container_name: rclone
    volumes:
        - './apps/rclone/config:/config/rclone'
        - '${ROOT}/mount:/data/mount:shared'
        - '${ROOT}/upload:/data/upload:shared'
    logging:
        driver: json-file
        options:
            max-size: 5m
    security_opt:
        - apparmor:unconfined
    devices:
        - /dev/fuse
    restart: always
    image: rclone/rclone
    command: 'mount --allow-other --use-mmap --buffer-size 0M union: /data/mount --allow-non-empty'

I then have another container that accesses the same destinations:

    - '${ROOT}/mount:/data/mount:shared'
    - '${ROOT}/upload:/data/upload:shared'

When moving something from within that container from above the mount folder into mount it always seems to do a copy instead of a move. I am switching from UnionFS where the same setup seemed to work ok.

Does RClone support moving instead of copy? Is it related to hardlinks and is that supported? I am getting the hardlink info from the threads here on the forum during the inception of the union part of RClone which suggests it does and from @Animosity022 's posts, seems relevant.

And is the config upstreams = /data/upload encgoogle::ro the equivalent to the old union method? I am trying to achieve all changes in /data/mount to take place in /data/upload and not to touch encgoogle:

2

The mount is effectively a new file system so it will always be a copy rather than a move into it.

You should be able to move things within the mount just fine.

I think that looks about right, except I'd put them in the other order encgoogle::ro /data/upload as the union backend always used to look at the last one first. I'm not sure it still does that without looking at the code, but there you go!

3

Most helpful, thanks.

On the always copy instead of move issue, I remember now how I used to manage it with UnionFS. I would create /data/upload/downloads so that the downloads folder is under the mount. I would then write directly to /data/upload/downloads to avoid having to handle the filesystem. Then when I did the move, I would move it from /data/mount/downloads/my-file, (which is effectively the same content as /data/upload/downloads) in to /data/mount/my-content/my-file. This way, UnionFS would see the move as being from within the same filesystem and would move instead of copy. For some reason though, this behaviour isn't replicated on the Rclone Union.

*edit: Although when I did it before, I wasn't using Rclone Docker to create the mount, I created it with Rclone installed directly on the system (although still used a container to access the mount and do the move) which may have also been a factor.

4

Ah, I see.

I think unionfs lets the underlying file system show through as it were, whereas rclone doesn't as some of the filesystems might not be local. Something like that!

5

This topic was automatically closed 3 days after the last reply. New replies are no longer allowed.