SSH_FX_FAILURE when moving file between folders in the same mount

dantebarba · August 21, 2020, 5:33pm

What is the problem you are having with rclone?

I currently have setup a openssh-server to serve two docker volumes as a sftp server. I then mount the root directory that contains the two volumes in a remote node using rclone mount. This is all done using Docker.

The structure looks like this:

ssh-server
-- vfs/
---- vfs/volume1
---- vs/volume2
rclone-mount
-- remote:/vfs local:/vfs

Then I'm trying to execute:

mv /vfs/volume1/afile /vfs/volume

This should allow me to do a server-side-move between volumes from a remote. Unfortunately, this command returns "SSH_FX_FAILURE".

I should clarify that this only happens when mounting a dockerized ssh-server. If I connect rclone to the standalone VPS ssh server using root the issue disappears. But I see no reason why it shouldn't work using docker in both ends. Executing the same move command by logging to the ssh terminal works fine too. So it's something between rclone/rclone and linuxserver/openssh-server that it's not working. Maybe some kind of ssh option that I've missed to enable?.

What is your rclone version (output from `rclone version`)

Version "v1.52.3"

Which OS you are using and how many bits (eg Windows 7, 64 bit)

Debian 9.

Which cloud storage system are you using? (eg Google Drive)

Private VPS.

The command you were trying to run (eg `rclone copy /tmp remote:tmp`)

mount Crown-vfs:/vfs /vfs --dir-perms 755 --file-perms 755  --gid ${PGID} --uid ${PUID} --allow-other --buffer-size 32M --poll-interval 1m --log-file=/config/rclone/logs_vfs/rclone.log --log-level=DEBUG

SSH-Server compose file:

  vfs-server:
    image: linuxserver/openssh-server:latest
    container_name: vfs-server
    restart: unless-stopped
    volumes:
      - ${STORAGE_LOCATION}/transmission/downloads/complete:/vfs/downloads
      - ${STORAGE_LOCATION}/radarr/movies/local:/vfs/movies
      - ${STORAGE_LOCATION}/vfs-server:/config
    environment:
      PGID: ${PGID}
      PUID: ${PUID}
      USER_NAME: ${VFS_USERNAME}
      USER_PASSWORD: ${VFS_PASSWORD}
      SUDO_ACCESS: 'true'
      PASSWORD_ACCESS: 'true'
    ports:
      - 2222:2222

Rclone-VFS compose file:

 rclone-vfs:
    image: rclone/rclone:latest
    container_name: rclone-vfs
    restart: unless-stopped
    privileged: true
    command: mount Crown-vfs:/vfs /vfs --dir-perms 755 --file-perms 755  --gid ${PGID} --uid ${PUID} --allow-other --buffer-size 32M --poll-interval 1m --log-file=/config/rclone/logs_vfs/rclone.log --log-level=DEBUG
    volumes:
      - "${STORAGE_LOCATION}/radarr/movies/remote:/vfs/movies:shared"
      - "${STORAGE_LOCATION}/transmission/downloads/complete:/vfs/downloads:shared"
      - "${CONFIG_LOCATION}/rclone:/config/rclone"
    environment:
      PUID: 1000
      PGID: 1000
    devices:
      - /dev/fuse
    cap_add:
      - SYS_ADMIN

The rclone config contents with secrets removed.

[Crown-vfs]
type = sftp
host = my.host.com
pass = alonghashforamaybesecurepassword
port = 2222
user = ssh-user
md5sum_command = md5sum
sha1sum_command = sha1sum

A log from the command with the `-vv` flag

2020/08/20 22:04:02 ERROR : downloads/The.Matrix.1999.REMASTERED.720p.BluRay.999MB.HQ.x265.10bit-GalaxyRG.mkv: Couldn't move: Move Rename failed: sftp: "Failure" (SSH_FX_FAILURE)
2020/08/20 22:04:02 ERROR : movies/The.Matrix.1999.REMASTERED.720p.BluRay.999MB.HQ.x265.10bit-GalaxyRG.mkv: File.Rename error: Move Rename failed: sftp: "Failure" (SSH_FX_FAILURE)
2020/08/20 22:04:02 ERROR : downloads/The.Matrix.1999.REMASTERED.720p.BluRay.999MB.HQ.x265.10bit-GalaxyRG.mkv: Dir.Rename error: Move Rename failed: sftp: "Failure" (SSH_FX_FAILURE)

asdffdsa · August 21, 2020, 5:39pm

hi,
i searched the forum for SSH_FX_FAILURE and found a solution for you
you need to use the latest beta version at https://beta.rclone.org/

dantebarba · August 21, 2020, 6:07pm

Yeah but I don't get that level of verbosity in my log. There are a lot of reasons why a SSH_FX_FAILURE can be generated. I'm sorry but I'm not convinced that we are talking about the same thing.

asdffdsa · August 21, 2020, 6:09pm

have you tried the beta, did it fail to fix the problem?

dantebarba · August 21, 2020, 6:12pm

switched from latest to beta
docker-compose down
docker-compose pull
docker-compose up -d
tested: error.

asdffdsa · August 21, 2020, 7:27pm

the beta is always the first step as it resolves many issues.

no worries, one of our many experts will stop by soon.

dantebarba · August 22, 2020, 12:05am

Tried using rclone move directly. Same error:

2020/08/21 21:02:17 DEBUG : Using config file from "/config/rclone/rclone.conf"

2020/08/21 21:02:22 DEBUG : sftp://PROTECTED: Using server side directory move

2020/08/21 21:02:22 INFO : sftp://PROTECTED//vfs/downloads: Server side directory move failed - fallback to file moves: can't copy directory - destination already exists

2020/08/21 21:02:24 INFO : sftp://PROTECTED//vfs/downloads: Waiting for checks to finish

2020/08/21 21:02:24 INFO : sftp://PROTECTED//vfs/downloads: Waiting for transfers to finish

2020/08/21 21:02:25 ERROR : The.Matrix.1999.REMASTERED.720p.BluRay.999MB.HQ.x265.10bit-GalaxyRG[TGx].en.srt: Couldn't move: Move Rename failed: sftp: "Failure" (SSH_FX_FAILURE)

2020/08/21 21:02:27 ERROR : The.Matrix.1999.REMASTERED.720p.BluRay.999MB.HQ.x265.10bit-GalaxyRG[TGx].es.srt: Couldn't move: Move Rename failed: sftp: "Failure" (SSH_FX_FAILURE)

2020/08/21 21:02:27 ERROR : The.Matrix.1999.REMASTERED.720p.BluRay.999MB.HQ.x265.10bit-GalaxyRG[TGx].mkv: Couldn't move: Move Rename failed: sftp: "Failure" (SSH_FX_FAILURE)

asdffdsa · August 22, 2020, 12:15am

what version of rclone?
what rclone command?

ncw · August 22, 2020, 10:42am

Unfortunately the SSH_FX_FAILURE error code is pretty vague!

If you could make a simple example with rclone move and show the command plus a log with -vv that would be really helpful.

dantebarba · August 23, 2020, 4:28am

I switched to rclone serve sftp. Now I can finally move! BUT I can't mount :(. When I mount the directory it's empty

I even tried logging in both containers:

created a directory in the sftp server
pointed the sftp mount to the created folder
logged in the sftp mount container and executed ls. No luck, the directory is empty.

So it looks like its not mounting anything, it's very weird. The logs only point out the following:

2020/08/23 04:20:24 DEBUG : : Root: 

2020/08/23 04:20:24 DEBUG : : >Root: node=/, err=<nil>

2020/08/23 04:20:24 DEBUG : /: Lookup: name="MAILPATH"

2020/08/23 04:20:24 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory

2020/08/23 04:20:25 DEBUG : /: Attr: 

2020/08/23 04:20:25 DEBUG : /: >Attr: attr=valid=1s ino=0 size=0 mode=drwxr-xr-x, err=<nil>

2020/08/23 04:20:25 DEBUG : /: ReadDirAll: 

2020/08/23 04:20:25 DEBUG : /: >ReadDirAll: item=0, err=<nil>

2020/08/23 04:20:25 DEBUG : /: Lookup: name="MAILPATH"

2020/08/23 04:20:25 DEBUG : /: >Lookup: node=<nil>, err=no such file or directory

Just to clarify, I've never created a folder or file called "MAILPATH". don't know where does it come from.

The new compose file for the VFS server using rclone serve:

  vfs-server:
    image: rclone/rclone:latest
    container_name: vfs-server
    restart: unless-stopped
    command: serve sftp /vfs --addr 0.0.0.0:2222 --gid ${PGID} --uid ${PUID} --user ${VFS_USERNAME} --pass ${VFS_PASSWORD} 
    volumes:
      - ${STORAGE_LOCATION}/transmission/downloads/complete:/vfs/downloads
      - ${STORAGE_LOCATION}/radarr/movies/local:/vfs/movies
    environment:
      PGID: ${PGID}
      PUID: ${PUID}
      TZ: America/Argentina/Buenos_Aires
      USER_NAME: ${VFS_USERNAME}
      USER_PASSWORD: ${VFS_PASSWORD}
      SUDO_ACCESS: 'true'
      PASSWORD_ACCESS: 'true'
    ports:
      - 2222:2222

The client:

  docker-vfs:
      image: rclone/rclone:beta
      container_name: docker-vfs
      restart: unless-stopped
      privileged: true
      command: mount Crown-vfs:/data /data --dir-perms 755 --file-perms 755  --gid ${PGID} --uid ${PUID} --allow-other --log-file=/config/rclone/logs_vfs/rclone.log -vvvv
      volumes:
        - "${STORAGE_LOCATION}/vfs-test/vfs:/vfs:shared"
        - "${CONFIG_LOCATION}/rclone:/config/rclone"
      environment:
        PUID: 1000
        PGID: 1000
      devices:
        - /dev/fuse
      cap_add:
        - SYS_ADMIN

ncw · August 23, 2020, 2:04pm

The mount is working but the root directory looks empty.

Does it look empty on the host?

dantebarba · August 23, 2020, 5:09pm

No, it does not.

ncw · August 23, 2020, 5:11pm

So when you enter the docker container and list the mount point, it doesn't look empty?

dantebarba · August 23, 2020, 5:24pm

Here is a video trying to explain the issue. On the left terminal is the VFS-Server on the right is the VFS-Client. The client is mounting the directory /vfs.

The test is simple. Both movies and downloads directory appear in the host SFTP server, but they are not present in the mount. Even if I create a new directory inside the container it does not show up.

system · October 23, 2020, 1:24pm

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.