Odd error about which user is running a postgres container

Help and Support
1

What is the problem you are having with rclone?

Postgresql container (part of LibrePhotos) complains that its data directory (the container-internal one) has "wrong ownership" when the volume for the directory is set to be on an rclone sftp mount (Hetzner storage box).

Rclone mount is successful, the problem is the container throws the following error:

initdb: removing contents of data directory "/var/lib/postgresql/data"
running bootstrap script ... chmod: changing permissions of '/var/run/postgresql': Operation not permitted
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
2023-03-22 09:16:07.003 UTC [28] LOG:  could not link file "pg_wal/xlogtemp.28" to "pg_wal/000000010000000000000001": Function not implemented
2023-03-22 09:16:07.006 UTC [28] FATAL:  could not open file "pg_wal/000000010000000000000001": No such file or directory
child process exited with exit code 1
initdb: removing contents of data directory "/var/lib/postgresql/data"

It appears the container isn't happy with the user owning the docker volume, however the user is me. If I try the same container with the volume mounted on the main storage of the VPS, the container works fine, it only crashes if the volume is on the rclone mount, so it must be the container doesn't like the way permissions/ownership is passed from rclone? Is there a better way to mount the storage box to get this container to work?

I did find that postgres containers have issues with Windows mounts, but this is not a Windows mount so not sure it's the same issue.

Run the command 'rclone version' and share the full output of the command.

rclone v1.62.2
- os/version: ubuntu 22.04 (64 bit)
- os/kernel: 5.15.0-67-generic (x86_64)
- os/type: linux
- os/arch: amd64
- go/version: go1.20.2
- go/linking: static
- go/tags: none

Which cloud storage system are you using? (eg Google Drive)

Hetzner Storage Box

The command you were trying to run (eg rclone copy /tmp remote:tmp)

rclone mount \
        --config /home/dinosm/.config/rclone/rclone.conf \
        --vfs-cache-mode full \
        --vfs-cache-max-age 30m \
        --vfs-cache-max-size 5G \
        --vfs-fast-fingerprint \
        --checkers 1 \
        --transfers 1 \
        --sftp-disable-concurrent-writes \
        --allow-other \
        --allow-non-empty \
        --umask 0027 \
        --uid 1001 \
        --gid 1002 \
        --log-file /home/dinosm/log/rclone.log \
        --log-level NOTICE \
        StorageBoxPhotosEnc: /media/StorageBoxPhotosEnc

The rclone config contents with secrets removed.

[StorageBoxPhotos]
type = sftp
host = <username>.your-storagebox.de
user = <username>-sub1
port = 23
pass = <Redacted>
use_insecure_cipher = true
shell_type = unix
md5sum_command = md5 -r
sha1sum_command = sha1 -r
concurrency = 2

[StorageBoxPhotosEnc]
type = crypt
remote = StorageBoxPhotos:
password = <redacted>
password2 = <redacted>

A log from the command with the -vv flag

2023/03/22 08:54:40 DEBUG : rclone: Version "v1.62.2" starting with parameters ["/usr/bin/rclone" "mount" "--config" "/home/dinosm/.config/rclone/rclone.conf" "--vfs-cache-mode" "full" "--vfs-cache-max-age" "30m" "--vfs-cache-max-size" "5G" "--vfs-fast-fingerprint" "--checkers" "1" "--transfers" "1" "--sftp-disable-concurrent-writes" "--allow-other" "--allow-non-empty" "--umask" "0027" "--uid" "1001" "--gid" "1002" "--log-file" "/home/dinosm/log/rclone.log" "-vv" "StorageBoxPhotosEnc:" "/media/StorageBoxPhotosEnc"]
2023/03/22 08:54:40 DEBUG : Creating backend with remote "StorageBoxPhotosEnc:"
2023/03/22 08:54:40 DEBUG : Using config file from "/home/dinosm/.config/rclone/rclone.conf"
2023/03/22 08:54:40 DEBUG : Creating backend with remote "StorageBoxPhotos:"
2023/03/22 08:54:40 DEBUG : StorageBoxPhotos: detected overridden config - adding "{vyqhZ}" suffix to name
2023/03/22 08:54:41 DEBUG : sftp://<username>-sub1@<username>.your-storagebox.de:23/: New connection [ipv6]:53986->[ipv6]:23 to "SSH-2.0-OpenSSH_8.8 FreeBSD-20211221"
2023/03/22 08:54:41 DEBUG : sftp://<username>-sub1@<username>.your-storagebox.de:23/: Shell type "unix" from config
2023/03/22 08:54:41 DEBUG : sftp://<username>-sub1@<username>.your-storagebox.de:23/: Relative path resolved to "/home"
2023/03/22 08:54:41 DEBUG : sftp://<username>-sub1@<username>.your-storagebox.de:23/: Using root directory "/home"
2023/03/22 08:54:41 DEBUG : fs cache: renaming cache item "StorageBoxPhotos:" to be canonical "StorageBoxPhotos{vyqhZ}:"
2023/03/22 08:54:41 INFO  : Encrypted drive 'StorageBoxPhotosEnc:': poll-interval is not supported by this remote
2023/03/22 08:54:41 DEBUG : vfs cache: root is "/root/.cache/rclone"
2023/03/22 08:54:41 DEBUG : vfs cache: data root is "/root/.cache/rclone/vfs/StorageBoxPhotosEnc"
2023/03/22 08:54:41 DEBUG : vfs cache: metadata root is "/root/.cache/rclone/vfsMeta/StorageBoxPhotosEnc"
2023/03/22 08:54:41 DEBUG : Creating backend with remote "/root/.cache/rclone/vfs/StorageBoxPhotosEnc/"
2023/03/22 08:54:41 DEBUG : fs cache: renaming cache item "/root/.cache/rclone/vfs/StorageBoxPhotosEnc/" to be canonical "/root/.cache/rclone/vfs/StorageBoxPhotosEnc"
2023/03/22 08:54:41 DEBUG : Creating backend with remote "/root/.cache/rclone/vfsMeta/StorageBoxPhotosEnc/"
2023/03/22 08:54:41 DEBUG : fs cache: renaming cache item "/root/.cache/rclone/vfsMeta/StorageBoxPhotosEnc/" to be canonical "/root/.cache/rclone/vfsMeta/StorageBoxPhotosEnc"
2023/03/22 08:54:41 DEBUG : Encrypted drive 'StorageBoxPhotosEnc:': Mounting on "/media/StorageBoxPhotosEnc"
2023/03/22 08:54:41 INFO  : vfs cache: cleaned: objects 0 (was 0) in use 0, to upload 0, uploading 0, total size 0 (was 0)
2023/03/22 08:54:41 DEBUG : : Root:
2023/03/22 08:54:41 DEBUG : : >Root: node=/, err=<nil>
2

Rclone doesn't support setting user id and group id for individual files yet. (Though the metadata framework is in to do this - we just haven't done it yet!)

You can use these flags on mount to specify uid and gid which you may be able to work around your problem with

  --gid uint32                             Override the gid field set by the filesystem (not supported on Windows) (default 1000)
  --uid uint32                             Override the uid field set by the filesystem (not supported on Windows) (default 1000)
3

Thank you for replying Nick. I am already using --uid and --gid on the mount command to set them to my user. The mount works correctly with the user I set, but the postgres container still complains the user is somehow wrong, even though when I set the container to use the local storage (owned by the same user), it works fine.

4

Log into the container and look to see what the permissions / user looks like within the container - that might throw some light on things.

5

The container keeps restarting, but I managed to log in enough to do an ls -l.
It turns out that the directory /var/lib/postgresql/data inside the container is owned by 1001:1002 (literally, it lists the owner as the number 1001), only when the container volume is mounted on the storage box via the rclone mount.

When it's mounted on the local storage, the owner of the same internal directory is postgres:root.

Initially I thought this was because of the --uid and --gid settings in the mount command. I removed them, but the internal dir still gets owned by 1001.

Would you have any idea what it is about my rclone mount that would effect this change in an internal container?

6

Upon further research, it must be the case (failing all else) that the Storage Box is seen as something akin to a Windows mount, therefore the postgres container fails due to a known issue with postgres containers and Windows mounts.

If that is the case, is there another way I could mount my storage box with rclone other than sftp, to make it look more like a standard linux storage area?

7

I would guess 1001 is your id (type the id command and see).

You need to find the id of the postgres user within the container and use that.

$ grep postgres /etc/passwd
postgres:x:124:134:PostgreSQL administrator,,,:/var/lib/postgresql:/bin/bash

so id 124 group 134 here.

I'm not sure exactly what the problem is here, so not sure what to recommend!

8

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.